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ABSTRACT 


DUDLEY KNOX LIBRARY 
NAVAL POSTGRADUATE SCHOOL 
MONTEREY CA 93943-5101 


The Janus simulation model was initially designed to operate in a stand-alone 
mode. There is an ongoing research project to link Janus to other constructive 
simulations and virtual simulators. The present standard used to connect different models 
is Distributed Interactive Simulation (DIS). Janus can operate in a DIS environment 
using a cell adapter unit called the World Modeler. The combination of Janus and the 
World Modeler is known as JLink. A goal of the JLink system is to replicate the 
analytical and training fidelity of stand-alone Janus in a distributed exercise. The 
purpose of this thesis is to assess the current state of JLink development. 

The experiment simulated three scenarios: armored, armored coalition, and light 
infantry battalions attacking against a defending company. All scenarios were executed 
in two contrasting environments. The simulation included the recently developed JLink 
features Family of Scatterable Mine (FASCAM) and chemical artillery. 

The thesis used five Measures of Performance to base the assessment: 1) 
FASCAM kills, 2) Chemical Artillery Kills, 3) Detection Ranges, 4) Kill Ranges, and 5) 
Loss Exchange Ratio. The statistical tests used for analysis were the Analysis of 
Variance (ANOVA) test, two-sample Mest, and Wilcoxon test. 

The results of the analysis show that JLink requires adjustments to artillery 
delivery methods in order to correct chemical artillery discrepancies and detection range 
issues. In general, JLink accurately portrays coalition warfare and satisfactorily replicates 
armored and infantry scenarios in contrasting environments. 
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EXECUTIVE SUMMARY 


The Army relies heavily on computer simulation as a training an analytical tool. 
One simulation the Army uses is Janus, a high resolution combat simulation model 
design to operate in a stand-alone mode. There is an ongoing research effort to link Janus 
with other constructive simulations or virtual simulators. The present standard for linking 
different models is Distributed Interactive Simulation (DIS). Janus can operate in a DIS 
environment using a cell adapter unit called the World Modeler. The combination of 
Janus and the World Modeler is known as JLink. A goal of JLink is to replicate the 
training and analytic fidelity of stand-alone Janus in a distributed (JLink) exercise. The 
level of fidelity may be gauged by comparing the results of a scenario run in stand-alone 
Janus with the results from the same scenario run in JLink. The purpose of this thesis is 
to assess the current state of JLink development by measuring the amount of distortion 
between stand-alone Janus and JLink. 

The experiment tested specific areas of JLink which had yet been assessed. 
Previously, only rudimentary scenarios had been tested in a single environment. This 
thesis tested scenarios using different types of combat units in two different 
environments. The thesis also tested JLink’s ability to portray coalition warfare. Finally, 
the thesis assessed recent software inputs to JLink which simulated specific combat 
functions, in particular Family of Scatterable Mines (FASCAM) and chemical artillery. 
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The experimental design used to assess these aspects of JLink included armored 
and light infantry units. The first scenario involved a Soviet-equipped armored battalion 
attacking against a defending US-equipped armored company. The second scenario was 
identical to the first except that the attacking battalion was divided into two separate 
forces, both attacking the armored company. This scenario was used to assess JLink’s 
ability to replicate a coalition battle. The last scenario involved a Soviet-equipped light 
battalion attacking against a US-equipped light infantry company. All three scenarios 
were executed in two different environments. The two environments selected were Fort 
Hunter Liggett, California (HL) and Southwest Asia (SWA). These two environments 
offered contrasting terrain, the first being wooded with rolling hills and the second being 
flat and open. Using different types of units in contrasting terrain permitted assessment 
of JLink’s ability to portray varied scenarios. To test the specific combat functions of 
FASCAM and chemical kills, the defending company used these weapons in every 
scenario. 

The Measures of Performance (MOPs) selected for analysis were FASCAM kills, 
chemical kills, detection range, kill range, rounds fired, and Loss Exchange Ratio (LER). 
These MOPs allowed for analysis of the basic procedures of an engagement. First, one 
entity must detect another entity. Once detected, the entity may elect to shoot at the 
target. Analyzing detection range, kill range, and rounds fired offers a quantitative 
method of assessing these basic engagement procedures. The LER is a function of the 
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other MOPs and measures the overall outcome of the battle. The MOPs FASCAM kills 


and chemical kills were used to assess JLink’s ability to simulate those functions. 

The statistical tests used to analyze the data were the Analysis of Variance 
(ANOVA), two sample r-test, and the nonparametric Wilcoxon test. First the data was 
analyzed using the ANOVA test whether overall averages for a specific MOP were 
statistically similar between the Janus and JLink runs taking into account all scenarios 
and environments. The ANOVA was also used to determine interactions between the 
scenarios, environments, and the mode (Janus or JLink) for each MOP. In the case where 
the ANOVA determined that the Janus and JLink means were statistically different, pair¬ 
wise two sample r-tests were used to identify where the differences in the means existed. 
Finally, the nonparametric Wilcoxon was also used in a pair-wise method to test those 
populations which did not satisfy r-test assumptions and to substantiate the Mest results. 

The results of the analysis show that JLink requires adjustments to artillery 
delivery methods in order to calibrate chemical artillery discrepancies and detection range 
issues. JLink chemical artillery killed far fewer entities than Janus, and JLink 
consistently detected farther than Janus. The finding is that to match Janus, JLink must 
simulate an artillery volley as one large cloud, like Janus, as opposed to individual 
rounds, as dictated by DIS standards. In general, JLink accurately portrays coalition 
warfare and satisfactorily replicates armored and infantry scenarios in contrasting 
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environments. 


XIV 


I. INTRODUCTION 


The Army relies heavily on computer simulations for training and force analysis. 
One simulation in use today is Janus, an event-driven wargame named for the two-faced 
Roman god Janus, who was the guardian of the portals of Rome and the patron of 
beginnings and endings. For nearly three decades, Janus has proven to be a valuable 
training and analysis tool and has evolved into a legacy model with many validated 
features. The current task is to link Janus, which is designed to operate in a stand-alone 
mode, with other live, virtual, or constructive models. 

The present standard used to connect two or more models is known as Distributed 
Interactive Simulation (DIS). Janus is capable of operating in a DIS environment using a 
cell adapter unit called the World Modeler. The World Modeler is software, designed to 
run on a low-end Silicon Graphics computer, which performs functions required by the 
DIS architecture that are not performed by Janus. The combination of Janus and the 
World Modeler is known as JLink [Ref. l:p.l]. 

JLink combines the training and analytic potential of a widely fielded constructive 
simulation, Janus, with a DIS-compatible simulation. A goal of the JLink system is to 
replicate the analytical and training fidelity of a stand-alone Janus in a distributed (JLink) 
exercise. This level of fidelity may be gauged by comparing the results of a scenario run 
in stand-alone Janus with the results of the same scenario run in JLink, using similar 
starting conditions. The purpose of this thesis is to assess the current state of JLink 
development by measuring the amount of distortion between stand-alone Janus and 
distributed Janus using JLink, by applying statistical analysis. The assessment will aid in 
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identifying credible areas of JLink as well as aspects of JLink which require further 
development. As interest in JLink continues to grow throughout the Army and across 
services, improvements in JLink fidelity will prove valuable and return many training and 
analytical benefits. 

A. JANUS 

1. Janus Background 

Lawrence Livermore National Laboratory originally developed Janus in 
the 1970s, with the current version developed by Training and Doctrine Command 
Analysis Center (TRAC) at White Sands Missile Range, New Mexico. Janus is a 
stochastic, interactive, high resolution, multisided, model with a robust database and a 
detailed post-processor for gathering results. 

Being a stochastic model, Janus uses a random number generator and 
probabilities to determine the outcome of detections and engagements. Janus random 
number seeds range from 1 to 99. Prior to executing a scenario, the user may set the 
random number seed manually or elect to have Janus randomly select the seed. Since 
Janus is an event based model, the user can achieve exact replication if the same random 
number seed is selected and there is no human interaction. 

The interactive nature of Janus permits its users to make decisions during 
the scenario to influence the outcome of the battle. Due to the variability of user inputs 
between scenarios, interaction will produce different results for two Janus scenarios using 
an identical random number seed. Examples of interactive functions include creating or 
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altering movement routes, planning and firing artillery missions, and 
mounting/dismounting units. User interaction is not necessary for pre-planned scenarios 
designed to fight two opposing forces. For purposes of this thesis, all scenarios were pre¬ 
planned and included no interaction. 

Janus is a high resolution model because it represents entities down to 
individual systems. However, when practical the user may aggregate entities, with 
artillery systems typically being aggregated up to 40 entities [Ref. 2:p.21]. The situation 
and scenario will generally dictate the level of aggregation, with Janus users often 
aggregating to the lowest useful level in order to take advantage of Janus’ high-resolution 
capabilities. 

Janus can also model up to six different sides for a given scenario, to 
include simulating fratricide. Janus offers the option to separate the sides among 
different workstations (primarily for training) or play all sides on one workstation 
(primarily for analysis). 

Janus possesses a robust database which permits the user to define a weapon 
system extensively or capture the detailed factors required for the scenario. The database 
is divided into six main sections: systems, weapons, sensors, engineer data, weather data, 
and chemical and heat data. Each main section has numerous subsections, all requiring 
inspection prior to creating a scenario to ensure accurate information is provided. A 
separate Janus database Manager’s Manual is available and necessary for reference. 

Janus’ post-processor permits the analyst to gather detailed results from 
scenarios. The post-processor offers a variety of detailed statistics, including reports on 
the number of rounds fired, kill ranges, detection ranges, minefield crossings, chemical 
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casualties, and Loss Exchange Ratios (LER). The user has the option to select which 
reports will appear in the post-processor, which precludes generating unnecessary reports 
while focusing on the area of interest. The reports of interest for this thesis are those 
reports mentioned above. 

2. Janus Uses 

Originally designed as an analytical model, Janus is presently used in the 
Training, Exercises, and Military Operations (TEMO) and the Advanced Concepts 
Requirements (ACR) domains. Both domains participate as members of the 
configuration control board to ensure all of the users’ requirements are satisfied. 

a. Training 

Janus is best geared to train leaders at brigade level and below 
because of its high resolution capabilities. The ability to model down to an individual 
system enables scenarios to focus on company, platoon, or squad size elements. 
Commanders at brigade level can use Janus to train the leaders of these elements in the 
decision-making process. Janus’ interactive capabilities allow leaders to make decisions 
during an engagement, and then to observe the outcome of their actions. Following the 
scenario, the entire battle can be replayed using the Janus Analyst Workstation (JAWS) 
and assessed for after-action review purposes. 
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b. Analysis 


Janus is particularly suited for the analyst. Janus’ post-processor 
permits the analyst to gather results from scenarios for further study of topics such as new 
tactics, techniques, and procedures. The analyst creates scenarios using standard 
methods, and then compares the results to a scenario using newly proposed methods. 

The analyst also uses Janus to study the effects of a new or modified 
weapon system. Janus’ database is robust enough to capture or alter many aspects of a 
weapon system, including weapon range, detection range, sensor type, travel speeds, and 
crew size. These weapon systems can also be tested in various environments and weather 
conditions based on the weapon’s mission profile. 

As a stand-alone simulation, Janus has aided Army trainers and analyst for 
years. Janus must now move to the next level of simulation, the DIS environment, in 
order to remain beneficial to the Army of the 21*‘ century. 

B. DISTRIBUTED INTERACTIVE SIMULATION (DIS) 

1. DIS Background 

The DIS Master Plan defines DIS as “A synthetic environment 
within which humans may interact through simulation(s) and/or simulators at multiple 
networked sites using compliant architecture, modeling, protocols, standards, and 
databases” [Ref. 3:p. I-l]. Leading edge computer technologies and advanced 
communications networking allow for such an environment to exist. Essentially, DIS is 
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an environment which brings the power of the computer and communications together to 
benefit the trainer and analyst. 

The terms of DIS are defined as follows: Distributed refers to geographically 
separated simulations, each hosted on a computer and connected via networks to create a 
shared environment. Interactive is described as different simulations electronically linked 
to act together and upon one another. Simulation is divided into three categories: a) live 
simulation where actual equipment and soldiers are operating in the field, b) constructive 
simulation which includes wargames and models such as Janus, and c) virtual simulation 
which involves manned simulators interacting within a virtual reality environment [Ref 
3:p. 1-2-1-3]. 

The primary objective of the DIS program is to establish an architecture which 
permits the linkage of different simulation environments into a seamless synthetic world. 
The infrastructure brings together systems built for separate purposes, technologies from 
different eras, products from different vendors, and platforms from various services [Ref 
3:p. 1-2]. Janus, operating in a DIS environment at Ft. Hood, Texas, linked to a virtual 
AH-64 Apache simulator at Ft. Hood and a constructive ModSAF simulation at Ft. 

Knox, Kentucky is a realistic application of the DIS objective (described above). 

2. DIS Benefits 

Since the development of DIS in 1991 with the successful linking and 
interacting of the Simulation Network (SIMNET) program, observers have 
enthusiastically embraced the benefits of DIS. Specifically, a DIS environment can 
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benefit the Army and other services in the areas of training, combat development, and 
systems acquisition. 

In terms of training, commanders have greater potential to train as they fight. If a 
DIS capability is established between live and constructive and/or virtual simulations, 
training will more realistically mirror combat conditions. Another aspect of DIS training 
benefits is in the area of jointness. An objective of DIS is to link simulations across 
services. Such a link enhances Joint training and promotes increased Joint training 
activities. 

DIS provides the combat development community expanded capabilities. First, 
DIS provides the ability to introduce new simulations into the synthetic DIS environment, 
such as an individual soldier simulated as a virtual entity, with minimal impact on 
software. DIS also provides combat developers the capability to collect and record 
actions and reactions generated by humans with minimal interference from observers 
[Ref 3:p. III-4]. 

The benefit DIS brings to systems acquisition is reduced test and evaluation costs. 
By linking live, constructive, and virtual simulations, the acquisition life cycle can be 
reduced significantly, resulting in test and evaluation cost reductions. 

The focus of this thesis is evaluating Janus in a DIS environment. The value of 
Janus and the importance of DIS were the driving forces in the development of JLink, the 
method by which Janus operates in the DIS environment. The next section highlights the 
methods of JLink and the challenges faced by JLink. 
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C. JLINK 


1. JLink Background 

In 1993, TRAC Monterey began developing the JLink system to support the Anti- 
Armor Advanced Technology Demonstration (A2ATD). The A2ATD was charged with 
developing and demonstrating the use of DIS to evaluate anti-armor weapon systems on a 
combined arms task force. TRAC Monterey had the overall responsibility for building 
the systems, providing project management, and contributing functional area expertise. 
Other contributors to the JLink project include the Naval Postgraduate School (NPS), 
RAND, MITRE, and Rolands and Associates (R&A). NPS and RAND developed the 
interface between Janus and the World Modeler. RAND also conducted terrain 
conversions and modified Janus internal algorithms as necessary, while MITRE focused 
on implementing dead reckoning algorithms. [Ref. l:p. 1] 

Following the A2ATD, TRAC Monterey continued to develop JLink to serve the 
TEMO and ACR domains with future training and analytical requirements. To assess the 
state of JLink development, TRAC Monterey conducted a comparison study of Janus to 
JLink in the summer of 1996. The study applied a rudimentary scenario in a single 
environment to test basic engagement methodologies. The results of the study showed 
that JLink produced similar kill ranges and loss exchange ratios when compared to Janus, 
but statically different shot totals for the blue forces. The cause of the total shots 
discrepancy was identified as a z-coordinate issue with respect to aircraft, and the study 
concluded that a scenario fought in stand alone Janus is “closely matched” with the same 
scenario fought under JLink’s current configuration [Ref 4:p. 9-10]. 
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As TRAC continues to develop JLink, recurrent testing is necessary to assess 
JLink’s current state. As JLink evolves into a robust model, capable of simulating 
multiple battlefield phenomena, comparison studies between Janus and JLink must be 
diverse enough to test and analyze newly developed aspects of the system. This thesis 
conducts such a test by simulating large scale scenarios in multiple environments. The 
experiment also includes recently developed JLink functions such as Family of 
Scatterable Mines (FASCAM) and chemical artillery. 

2, JLink Architecture 

This section describes the JLink architecture, which enables Janus to 
operate in a DIS environment through use of the World Modeler. Network management 
establishes the connection between Janus and other DIS-compatible models, such as 
ModSAF, using two different protocols. The Transmission Control Protocol/Intemet 
Protocol (TCP/IP) is used to establish communications between the World Modeler and 
Janus, while the User Datagram Protocol (UDP/IP) is used to establish communications 
between the World Modeler and DIS. Through these links, the World Modeler controls 
dataflow by passing Protocol Data Unit (PDU) information to Janus and DIS at the 
appropriate time intervals [Ref. l:p. 5] (Figure 1). The purpose of the PDU is to 
facilitate the electronic transfer of data between simulations with different software. 
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Figure 1. JLink Design 


Suppose Janus is portraying Blue forces and a DIS-compatible simulation is 
portraying Red forces. When Blue generates an event, such as firing at a Red entity, 
Janus sends a message to the World Modeler via the TCP/IP connection. The World 
Modeler uses “hooks” to capture the Janus protocol and converts the Janus protocol to a 
DIS PDU, which can be processed by a DIS model. The World Modeler then broadcasts 
the DIS PDU via the UDP/IP connection to the network for use by the DIS (Red) 
simulation. Conversely, when the Red force generates an event, the corresponding DIS 
PDU is sent via the network to the World Modeler, where it converts the DIS PDU to a 
Janus protocol, which in turn is sent to the Janus simulation for processing. In essence. 
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one of the functions of the World Modeler is to convert Janus Protocols to DIS PDUs and 


vice versa. 

Currently, JLink has successfully interacted with several constructive simulations 
and virtual simulators, via DIS PDUs. However, evolving systems require extensive 
testing, and the JLink process frequently requires adjustments or model calibration to 
achieve effective levels of interoperability between interfacing systems. 


11 


12 


11. PROBLEM DESCRIPTION 


Improved interoperability can be translated into minimizing the distortion between 
results from Janus stand-alone and the results of the same scenario run in JLink. Two or 
more simulations/simulators are defined as “DIS interoperable” when their performance 
characteristics support a fair fight to the fidelity required for the exercise. A fair fight 
exists when the differences in two simulations’ performance are overwhelmed by user 
actions [Ref. 5:p. A-4]. There are two general factors which may contribute to 
differences in the results between Janus stand-alone and JLink scenarios and lead to an 
unfair fight. These factors are the JLink architecture and the DIS PDU itself, which form 
the basis for the problem description of this thesis. This chapter addresses how JLink 
architecture and the DIS PDU may contribute to distortion. As part of the problem 
description, the chapter concludes by identifying specific features of JLink, which may be 
affected by the factors mentioned above, that require testing. 

A. DISTORTION FROM JLINK ARCHITECTURE 

Inherent aspects of the JLink architecture may contribute to differences in 
JLink results versus Janus stand-alone results. Given that Janus is an event-driven model, 
the sequence of events is vital to the outcome. In stand-alone Janus, a scenario will 
produce the same sequence of events, and hence the same results, for every run as long as 
the random number seed remains unchanged and no human interaction occurs. In JLink, 
delays in passing and converting PDUs may produce a different sequence of events than 
the corresponding Janus stand-alone scenario. Delays in passing PDUs between DIS 
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models are a function of network congestion and quality of the network hardware. 

Delays in converting Janus protocols to DIS PDUs, and vice versa, also contribute to an 
altered sequence of events. A different sequence of events will result in different 
outcomes. Say for instance a Blue entity in stand-alone Janus generates a shot event on 
the event list against a Red entity at time =10 minutes, resulting in a Red kill at time = 11 
minutes. In JLink, the same shot event may not get scheduled until time =12 minutes 
due to delays in passing and converting the appropriate PDU, which may result in the Red 
entity surviving (and shooting) longer than in Janus stand-alone. 

Another aspect of JLink architecture contributing to distortion is the issue of 
random number generators. Janus stand-alone utilizes only one random number 
generator to determine the outcome of all stochastic events. JLink uses the Janus random 
number generator to adjudicate only those events corresponding to its own side. The 
other sides represented in the scenario use their own random number generators to 
adjudicate stochastic events. Even if all random number generators involved behave 
identically (which is unlikely) and they are all initiated with the same random number 
seed, they will not produce the same aggregate random number string as the Janus stand¬ 
alone generator for an entire scenario. Dissimilar random number strings will result in • 
dissimilar results between Janus stand-alone and JLink. 

This thesis will not attempt to determine whether any JLink architecture issues 
like those above contributed to distorted results. The thesis experiment did, however, 
reduce the impact of the architecture on results by using the following procedures. To 
reduce interference when passing PDUs, all JLink runs were executed during times when 
the TRAC Monterey network was free from all other activity. Also, all scenarios mn in 
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JLink used the same random number seeds used in Janus stand-alone. Using the same 
random number seeds will produce some of the same random numbers for both JLink and 
Janus stand-alone scenarios. 

B. DISTORTION FROM DIS PDUs 

One major challenge to World Modeler developers is to accurately translate and 
create DIS PDUs as they flow to and from the Janus model. The DIS PDUs created by 
the World Modeler for use by other distributed systems must portray the originating Janus 
protocol as accurately as possible. Likewise, DIS PDUs sent to the World Modeler by 
distributed simulations must be translated appropriately for use by Janus. Inaccurate or 
inappropriate conversions of PDUs may produce distorted JLink results as compared to 
the Janus stand-alone standard. 

One example where appropriate conversions of PDUs is critical is the flight 
altitude of rotary-wing aircraft. Janus only portrays two flight levels, treetop level (low) 
and at altitude (high), whereas ModSAF simulates continuous altitudes. The issue 
becomes at which flight level a ModSAF helicopter entity should be simulated in Janus, 
and at what altitude a low flying Janus helicopter should be simulated in ModSAF. 

In order to test for distortion caused by the DIS PDUs, JLink requires additional 
testing of a wide variety of features. To date, JLink fidelity has only been tested using 
rudimentary scenarios in a single environment. JLink development is now at the stage 
which can support testing of additional aspects of the system. 
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C. JLINK FEATURES TESTED IN EXPERIMENT 


1. Varied Scenarios 

One aspect of the JLink system this thesis tests is JLink’s ability to 
accurately portray different types of combat units in the offense or defense, negotiating 
obstacles, under different environmental conditions. Such scenarios have yet to be 
validated using JLink. If JLink operates as designed, the results from these varied 
scenarios will be similar to the results of the same scenarios run in stand-alone Janus. 

2. Coalition Warfare 

The thesis experiment also includes an assessment of JLink simulating the 
synergistic effects of coalition warfare. Previous tests involving JLink have only 
included two sides fighting against one another. As part of the experiment, one scenario 
includes three sides, two forces allied together fighting against the third. Pitting two sides 
against a lone third side may generate insightful results regarding JLink’s ability to 
accurately fight a coalition battle. 

3. Specific Combat Functions 

In addition, the thesis tests whether JLink accurately portrays the effects of 
specific combat functions. Recent JLink software inputs, designed to replicate specific 
combat functions such as chemical artillery delivery methods and FASCAM, require 
testing. In stand-alone Janus, chemical artillery volleys are delivered as an aggregated 
threat, with one large radius. DIS standards require that individual artillery rounds be 
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passed between simulations. Because of this requirement, programmers have modeled a 
JLink chemical artillery volley as having each round create its own radius of effect. 
Figure 2 depicts a volley of three artillery rounds and illustrates the difference in the two 
delivery methods. 

O 

Stand-Alone Janus JLINK 

Figure 2. Artillery Delivery Methods 

The thesis will determine whether such a distribution of chemical artillery rounds 
significantly affects the number of kills. 

Another specific combat function requiring testing is FASCAM, an 
artillery-delivered minefield. Unlike chemical artillery, aggregation is not an issue 
regarding FASCAM. This thesis provides initial feedback on the FASCAM code as to 
whether it generates the same number of kills as FASCAM in stand-alone Janus. 

Herein lies the goal of this thesis — to assess JLink fidelity by testing whether 
JLink produces similar results to stand-alone Janus using varied scenarios and coalition 
warfare, and by determining whether recent software inputs to specific combat functions 
are providing accurate representations. 
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III. DESIGN OF EXPERIMENT 


The overall goal of the experiment was to provide a means to obtain data from a 
scenario executed in Janus and the same scenario executed in JLink for purposes of 
analysis. The issues involved in meeting the goal were selecting appropriate data to be 
analyzed, determining the structure of the scenarios, and conducting the experiment. 

This chapter begins by presenting the measures of performance and the methodology 
supporting their selection. Next, details of the scenarios are described, followed by a 
discussion of the conduct of the experiment and issues encountered with the execution. 
The experiment included necessary conditions for testing the issues defined in Chapter II. 

A. MEASURES OF PERFORMANCE 

The first issue in designing the experiment was to identify what data would be 
analyzed. The MOPs were limited only by the information provided in the post¬ 
processor. For clarity, this thesis refers to the data as measures of performance, as 
opposed to measures of effectiveness. The reason for the distinction is that the purpose of 
the thesis is not to analyze the effectiveness of a new weapon system or perhaps a tactic. 
Rather, the focus is to assess JLink fidelity by comparing the performance of the JLink 
system to the performance of the stand-alone Janus system. 

The MOPs were derived from issues raised in the Problem Description. The areas 
that require testing in JLink are varied scenarios, meaning scenarios using different types 
of units in contrasting environments, coalition warfare, and specific combat functions 
such as chemical artillery and FASCAM. The determination whether JLink accurately 
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portrays the areas defined in the Problem Description is derived from the analysis of one 
or more of the following MOPs: 1) Detection Ranges, 2) Kill Ranges, 3) Rounds Fired, 
4) Loss Exchange Ratio (LER), 5) Chemical Artillery Kills, and 6) FASCAM Kills. 

1. MOPs for Varied Scenarios and Coalition Warfare 

To determine the fidelity of varied scenarios in JLink, the thesis analyzed 
detection ranges, kill ranges, rounds fired, and LER. Applying appropriate MOPs to the 
broad topic of scenario fidelity is critical to arriving at valid conclusions. The analysis 
must focus on the basic procedures of an engagement. First, one entity must detect 
another entity. Comparing the detection ranges of Janus stand-alone and JLink is a 
quantitative method of assessing the detection process in JLink. Next, the detecting entity 
may elect to shoot at the target, possibly resulting in a kill. Analyzing the kill ranges 
between Janus and JLink is a quantitative method for assessing the killing process in 
JLink. Since not all shots result in kills, the total number of rounds fired must also be 
analyzed. The post-processor also provides the flexibility to analyze detection ranges, kill 
ranges, and rounds fired for an entire scenario or by individual side. For instance, rounds 
fired can be analyzed by comparing total Janus rounds fired to total JLink rounds fired for 
the same scenario, or by comparing blue Janus rounds versus blue JLink rounds. 

Detecting and killing the target are the basic elements of a battle. If 
JLink replicates these functions in the likeness of Janus, one would expect similar overall 
outcomes between Janus stand-alone and JLink scenarios. Further, since LERs are 
primarily a function of detections and kills, the LERs offer a quantitative method to 
assess the overall outcome of a JLink scenario. 
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2. 


MOPs for Chemical Artillery and FASCAM 


To determine whether JLink achieves acceptable results from chemical 
artillery and FASCAM, the thesis analyzed the number of kills generated by chemical 
artillery and the number of kills resulting from firing FASCAM. 

B. SCENARIOS 

The scenarios were determined based on the Problem Description requirements. 
Specifically, the thesis must analyze results from varied scenarios, coalition warfare, and 
the specific combat functions of chemical artillery and FASCAM. US and Russian-style 
armored and light infantry units, along with their supporting elements, were a logical 
selection for the unit types. Armored and light infantry units are combat arms likely to be 
involved in offensive or defensive operations. Units equipped with US- and Russian- 
made weapons bring diverse capabilities to the battlefield, adding robustness to the study. 

In determining the environments, the goal was to select two contrasting locations 
which will likely have different effects on detection ranges and engagement ranges. The 
two contrasting environments used were Southwest Asia (SWA) and Hunter Liggett, 
California (HL). SWA is primarily desert with no trees and slightly undulating terrain, 
which provides some relief. HL is comprised of small hills and lightly wooded terrain. 
The relatively flat terrain of SWA compared to the rolling, wooded terrain of HL offers 
contrasting settings to test sensor functions and weapons platforms. 

After determining the types of units and environments, the final element in the 
scenario stmcture was the means of engagement. The engagement used was an attacking 
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battalion against a defending company. Such an engagement provided results from units 
with contrasting postures, moving as opposed to stationary. Different postures affect the 
detection and firing capabilities of entities in Janus. 

Once all of the elements of the scenarios were identified, three scenarios were 
developed. The first scenario involved a US-equipped armored company (blue) 
defending against an attacking Russian-equipped armored battalion (red). The second 
scenario was identical to the first, except that the red force was divided into two sides to 
form a coalition. The red coalition attacked the (defending) blue force to test the results 
of a coalition engagement, in both a stand-alone and distributed mode. The third scenario 
involved a US-equipped light infantry company (blue) defending against an attacking 
Russian-equipped light infantry battalion (red). In every scenario, the blue side fired 
chemical artillery rounds and FASCAM at specifically identified red units traveling on a 
narrow avenue of approach. This was done to isolate the weapons’ performance to aid in 
the deconfliction of MOPs for analysis. Also, all blue sides possessed obstacles to aid in 
the defense. The obstacles included minefields and abatis for road barriers. 

1. Scenario 1 

The weapon systems used by the blue side in scenario 1 were the MlAl 
tank, M2 Infantry Fighting Vehicle (IFV), Multiple Launcher Rocket System (MLRS) 
artillery, AH64 Apache helicopter, and four riflemen as scouts. The red side consisted of 
the T72 tank, BMP-2 IFV, MLR 16 (MLRS equivalent). Hind helicopter, and SAM air 
defense weapon. The blue side consisted of 32 total units (mainly MlAls), while the red 
side had 91 total units (mainly T72s), resulting in approximately a 3:1 ratio in favor of the 
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attacker. The scenario was designed to emphasize the capabilities of the Ml A1 and T72 
tanks, with support from the other systems. 

2. Scenario 2 

The weapon systems, weapon numbers, and scenario design in scenario 2 
were identical to those in scenario 1. In scenario 2, the red force was divided into two 
sides, fighting together as a coalition against the blue force. 

3. Scenario 3 

The weapon systems used by the blue side in scenario 3 were the rifleman with 
M16 rifle, rifleman with machine gun. Light Anti-tank Weapon (LAW), mortars, and 
MLRS. The red side consisted of the rifleman with AK47 rifle, rifleman with machine 
gun, anti-tank RPG, mortars, and MLR16. The blue side had 116 units versus 653 units 
for the red side, resulting in a 5.6:1 edge in favor of the attacker. The scenario 
emphasized the capabilities of the rifle and machine gun. 

4. Scenario General Characteristics 

Red forces attacked from north to south in all scenarios since likely avenues of 
approach in both environments favored north-south attack routes. Recall that blue forces 
fired chemical rounds at specific red units to assess their effectiveness in distributed 
simulations. Since the wind generally travels from southeast to northwest in both 
environments, it was logical to have the blue side firing northward, or downwind. Hence, 
the red forces were arrayed to attack south. 
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Another common characteristic between scenarios was the placement of the blue 
forces. All blue units were placed on relatively defensible terrain, oriented to the north. 
Ensuring the blue units had adequate Line of Sight (LOS) to the north was critical to the 
defense. Figure 3 shows the Janus map of HL, with the blue units in position. The figure 
also shows the LOS of one blue entity, with “blind spots” shown as broken radials. 



Figure 3. Scenario Graphics 


Additionally, the scenarios were designed so that all blue entities could fire at all 
red entities, and vice versa. This design required slight adjustments to the database in 
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some instances. The purpose of this design was to test JLink’s ability to portray multiple 
types of weapons engaging each other. 

Finally, all scenarios lasted between 17-27 minutes, with the average length being 
approximately 20 minutes. Although the test scenario is short compared to typical Janus 
exercises, which often last several days, 20 minutes permitted artillery units to fire 
assigned volleys, “pucked” FASCAM to fire five minefields, aircraft to fly designated 
routes, and attacking elements to move into and through the engagement area. 

C. EXPERIMENT 

1. Conduct of Experiment 

Each of the three scenarios were executed in two environments, SWA and HL, 
resulting in six combinations. Each combination was then run in two different modes. 
First they were run in the Janus stand-alone mode, Janus (blue) versus Janus (red), with 
both the blue and red forces simulated on the same Hewlett-Packard (HP) workstation. 
Then the same six combinations were run in the JLink (blue) versus JLink (red) mode, 
with the two forces simulated on separate workstations, as illustrated in Figure 4. The 
JLink versus JLink mode is the method developed at TRAC Monterey to test distributed 
Janus without having to connect to another type of DIS simulation. Each Janus terminal 
is connected to a World Modeler which in turn is connected to the network. 


25 



JUNK 


Figure 4. JLink vs JLink 


Six combinations run in two different modes resulted in 12 total combinations in 
order to capture all scenarios, environments, and modes. Figure 5 depicts the overall 
experiment. 



Janus Stand-Alone 

JLink vs JLink 

Armored SWA 
Armored HL 

(avg of 10 runs) 
(avg of 10 runs) 

(avg of 10 runs) 
(avg of 10 runs) 

Armored Coalition SWA 
Armored Coalition HL 

(avg of 10 runs) 
(avg of 10 runs) 

(avg of 10 runs) 
(avg of 10 runs) 

Light Infantry SWA 
Light Infantry HL 

(avg of 10 runs) 
(avg of 10 runs) 

(avg of 10 runs) 
(avg of 10 runs) 


Figure 5. Overall Experiment (Averages for Specific MOP) 


These 12 combinations, however, only provide one data point for each 
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MOP, and will be referred to as a single run of the experiment. Ultimately, 10 runs were 
performed for each cell to provide a reasonable sample. (The issue regarding the total 
number of runs is addressed in Chapter III, paragraph C. 2.) 

The desired method of analysis was to compare the MOPs resulting from a 
specified number of Janus stand-alone runs to the MOPs resulting from the same number 
of JLink runs. Since a stand-alone run will produce the exact same results if the same 
random number seed is sown, all stand-alone runs were executed using different, 
randomly selected, seeds. The same random number seeds were then used in the 
corresponding JLink runs. Also, due to the inherent variability caused by JLink 
architecture, 10 additional runs were performed in JLink using the same random number 
seed for all 10 runs to isolate the variance contribution from that aspect of the JLink 
system. 


2. Experiment Issues 

As mentioned previously, all JLink runs were done during times of minimal traffic 
on the TRAC Monterey network. In all but four JLink runs, the scenarios were executed 
when the network was completely free of other users. Executing the scenarios with 
minimal network interference reduces variability in the results due to event scheduling. If 
the network is congested, delays may occur in passing information and the sequence of 
events may be affected. 

Determining the sample size is fundamental to the design of every experiment. 

An experiment is conducted a number of times so that the data produces good estimators 
of the true population parameters. Different techniques are available to arrive at a 
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satisfactory sample size. The thesis proceeded as follows regarding this issue. First, a 
practical approach was used. Based on experience and recommendations from senior 
Janus analysts, a sample size of n = 10 generally produces results with “acceptable 
variance” for the MOPs defined in this thesis. That is, the experiment is run until an 
estimate of the variance for the mean is reduced to a pre-determined, acceptable level. 
This level will be different for each measure of performance. Based on the raw data 
figures in Appendix A, acceptable variance was achieved in detection ranges, kill ranges, 
LERs, and for most cases of rounds fired. Discussion in Chapter IV highlights anomalies 
in FASCAM and chemical kills which preclude the possibility of reaching acceptable 
variances levels for these MOPs with reasonable sample sizes. 

The thesis also considered the issue of normality of the data when 
determining sample size. Normality of the data set is one assumption in the two sample t- 
test, which is used for analysis of the MOPs. The normal distribution quantile plots in 
Appendix F show that the results for most MOPs are relatively normal, with some 
obvious exceptions. Also, the r-test normality assumption is generally satisfied if the 
sample averages are normally distributed. As the sample size increases, one would 
expect the sample averages to become more normal, based on the Central Limit Theorem 
[Ref. 6:p. 232]. In this case, we expect a sample size of n=10 to be large enough to 
produce normally distributed averages. 

Another issue addressed prior to beginning the runs was that the user can only fire 
FASCAM while the scenario is on-going. The users timing and accuracy could vary 
significantly from run to run and thus alter the FASCAM results. The solution was 
discovered in the “puck” run. In essence, a puck run records all of the actions of a 
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previously run scenario, including the user actions, and replays them on the current 
scenario run. By pucking the initial run of each scenario, then changing the random 
number seeds for the following runs, the user can replicate the same scenario, to include 
FASCAM volleys, while generating different results due to the new seed. 

The final issue encountered in conducting the experiment occurred after the runs 
were complete. The issue involved gathering the information from the post-processor 
into a usable form. After transferring the post-processor information to a personal 
computer, extensive manipulation of the data was required prior to importing the data 
into a spreadsheet or statistical package. Since the results from the post-processor are 
lengthy and involve thousands of numbers in the case of these scenarios, manipulating the 
data was cumbersome and occasionally resulted in lost data. To resolve the issue, a C 
program was written and used as a filter to easily gather the required data for the MOPs. 
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IV. DATA ANALYSIS 


This chapter presents the methodology for analyzing the results, the statistical 
tests used in the analysis, and the results from the analysis. The following chapter 
provides a detailed analysis of the results presented in this chapter. 

A. ANALYSIS METHODOLOGY 

After running the 12 combinations described in Figure 5, with 10 replications for 
each cell, summary statistics for each MOP were gathered. The raw data for the 
experiment is presented in Appendices A, B, and C. The LER is the total red entities 
killed divided by the total blue entities killed. Each data entry for detection range and kill 
range is the average of all detection ranges (kill ranges) for the specified mode on the 
given run. The rounds fired data is the total of all direct fire rounds shot for the specified 
mode on the given run. 

The data was then analyzed by MOP. First, the Analysis of Variance (ANOVA) 
test was used to determine if differences existed between the means of the Janus 
population and the JLink population for the given MOP. ANOVA determines whether 
sample means from different populations are statistically similar by analyzing the ratio of 
the variance estimates. ANOVA also helps identify interactions occurring between the 
three factors of the experiment - scenario type, environment, and mode (Janus/JLink) - 
with respect to the given MOP. 

The ANOVA test identifies whether their are differences in the means between 
the two populations, but does not indicate where the difference occurred. To identify 
where the differences in the means exist, a pair-wise two sample /-test was applied to 
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those MOPs whose populations were deemed statistically different by the ANOVA. In 
this case, the goal was to determine whether the results, from an MOP in a specific 
scenario and environment executed in Janus stand-alone, are statistically similar to the 
results of the same MOP under the same conditions executed in JLink. Such a pair-wise 
comparison highlights where differences in the means exist. 

For this thesis, the sample size is relatively small and the population variances are 
unknown, hence a two sample t-test is appropriate in this case. [Ref. 6:p. 357-359]. The 
assumptions for the two sample r-test are: 

1. Both samples are independent random samples from normal populations. 

2. The two populations have equal variance. 

Data sets were analyzed to determine whether assumptions were satisfied prior to 
applying the t-test. 

In cases where the assumptions were not satisfied, the data was analyzed in the 
same pair-wise method using the less powerful Mann-Whitney-Wilcoxon (Wilcoxon) 
nonparametric test. The Wilcoxon test does not require that the data satisfy the normality 
assumption above. The Wilcoxon test was also performed on the MOPs which did satisfy 
the assumptions to further substantiate the results of the t-test. 

The data analysis concludes with a discussion of the variability within the JLink 
system to determine if the main contributor to variance comes from within the JLink 
system or from within the simulation runs. 
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B. TOOLS FOR ANALYSIS 


1. ANOVA 

The ANOVA determines whether sample means from different populations are 
statistically similar and identifies interaction between factors. The design of the 
experiment resulted in the data being organized into a 3 x 2 x 2 fashion, meaning the 
experiment tested three factors, the first involving three levels, and the second and third 
factors involving two levels (Figure 5). The first factor, scenario type, was tested at three 
levels: armored, armored coalition, and light infantry. The second factor, environment, 
was tested at two levels: SWA and HL. The third factor, mode, was tested at two levels: 


Janus stand-alone and JLink. The first and second factors were blocking factors, while 


the third factor acted as the treatment factor which this thesis intended to analyze. 


An interpretation of the ANOVA table follows. Table 1 shows the results of the 


ANOVA for detection ranges by the blue forces. 


Analysis of Variance Table for Detection Range Blue 
Response: MOE (Detection Range Blue) 

Terms added sequentially (first to last) 



Df 

Sum of Sq 

Mean Sq 

F Value 

Pr (F) 

Seen 

2 

95.34164 

47.67082 

6050.279 

0.0000000 

Env 

1 

2.51797 

2.51797 

319.576 

0.0000000 

Mode 

1 

4.11438 

4.11438 

522.188 

0.0000000 

Seen:Env 

2 

0.19275 

0.09638 

12.232 

0.0000163 

Seen .-Mode 

2 

1.64627 

0.82313 

104.470 

0.0000000 

Env:Mode 

1 

0.09369 

0.09369 

11.892 

0.0008050 

Seen:Env:Mode 

2 

0.00861 

0.00431 

0.547 

0.5805110 

Residuals 

108 

0.85094 

0.00788 




Table 1. ANOVA Results for Blue Detection Ranges 


The ANOVA table provides results for analysis of blocking factors, scenario type and 


environment, and the treatment factor, mode (Janus/JLink). The focus of the thesis is on 


the treatment factor, mode, to assess whether Janus and JLink produce similar results for 
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given MOPs. The highlighted row, labeled Mode, is interpreted as follows. The 
ANOVA model equation for , the observation recorded for the /th scenario, and jih 

environment, and kth mode, is 

=jU + A+1^;+r*+£«, 

where fi is the overall mean 

A is the effect of the ith treatment 
Tj is the effect of the yth environment 
is the effect of the /dh mode 
is the experimental error ~ Normal (0,cr^), 

and it is initially assumed that there is no interaction between factors [Ref. 7: p. 532]. In 
words, the recorded observations for Blue Detection Ranges, separated by mode, 
(highlighted row above) are interpreted as follows: 

Janus Blue Det Rng = overall mean Blue Det Rng + ScenEffect + EnvEffect + JanusModeEffect + error. 
Similarly, every JLink recorded observation is interpreted as follows: 

JLink Blue Det Rng = overall mean Blue Det Rng + ScenEffect + EnvEffect + JLinkModeEffect + error. 
Assuming the model is additive, subtracting the JLink equation from the Janus equation 

returns a model which isolates the MOP in terms of the mode effects (and error): 

Janus MOP - JLink MOP = Janus Mode Effect - JLink Mode Effect + error. 

In this case, the zero p-value in the Mode row means that the null hypothesis 

: mean Janus Blue Det Rng = mean JLink Blue Det Rng 
is strongly rejected. The ANOVA does not, however, identify where the difference 
occurred. 
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Figure 6 shows interaction plots for scenario;environment, scenarioimode, and 
environment:mode for the same MOP, blue detection ranges. The interpretation of the 


scen:mode interaction is as follows. The null hypothesis tested is 




AvgJanusBlueDetRng — AvgJLinkBlueDetRng = A,scen#l 
AvgJanusBlueDetRng — AvgJLinkBlueDetRng = A,scen#2 
AvgJanusBlueDetRng — AvgJLinkBlueDetRng = A,scen#3 


yAllA s the same. 


In this case, the p-value for scen:mode interaction is zero, thus rejecting the null 
hypothesis. This result can be interpreted as saying, not only are the sample means for 
blue detection ranges different between Janus and JLink, the difference varies between 
scenario. Figure 6 highlights with arrows where the difference in means are considerably 
larger between Janus and JLink for scenario #1 compared to the difference in scenario #3. 



Figure 6. Interaction Plots for Blue Detection Ranges 

The value of the interaction plots for this thesis is that they provide insights as to 
the cause of disparate results between Janus and JLink. For instance, the scenrmode 
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interaction plot shows that greater differences in blue detection ranges occur in armored 
and armored coalition scenarios as compared to light infantry scenarios. This finding 
indicates that situations peculiar to armor and armor coalition scenarios contribute most 
to the gap in blue detection ranges between Janus and JLink. 

2. Two Sample f-Test 


a. Test Description 


The two sample /-test tests the following hypothesis: 

=A, 

versus ^ A. 

In this case, the thesis determines whether the two sample averages are statistically 
similar, so our A = 0. The test statistic used is 


/ = 


X - y- A 



where s^, is the pooled sample variance, defined at the beginning of this chapter, and x 
and y are the two sample averages. The test statistic is then compared to the critical 
value for a two-sided test. The null hypothesis is rejected if 

^ - ^o/2.2«-2 Of ^ - “^a/2,2n-2 • 

One method used to determine the confidence level a, when several /- 
tests are conducted, is the Bonferroni method. Essentially, the Bonferroni method divides 
the original a value, in this case 0.05, by the total possible number of /-tests conducted 
[Ref. 8:p. 424]. This method offers a technique for placing a tighter restriction on the 
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rejection region of the null hypothesis, which would be otherwise rejected 5% of the time 
simply by chance if the original a was used. When the t-test was applied to an MOP, six 
different pair-wise tests were necessary to encompass three scenarios executed in two 
environments. Since a total of 6 comparisons were conducted, the new confidence level 
used for these r-tests was 

a = a ,,/6 = 0.0083. 

b. Methods for Testing Assumptions 

Before applying the two sample /-tests, the assumptions of normality and 
common variance between sample populations were considered. Two tests were 
conducted to determine if the assumption of normality was satisfied. First, Quantile- 
Quantile Plots (Q-Q Plots) for the normal distribution were plotted. The results are 
shown in Appendix F. The population is considered normally distributed if the plot 
produces a generally straight line. 

The Kolmogorov-Smirnov (KS) Goodness of Fit (GOF) was also used to 
test for the normality assumption. The test determines if a sample population follows a 
prescribed distribution. The KS GOF tests the following hypothesis: 

H^: Sample Population ~ Normal (x,s^), versus 
H True cdf does not equal normal distribution for at least one sample point. 

For small sample sizes, such as 10, the KS GOF rarely rejects the null hypothesis at a 
a=0.05 level of significance. Because of this fact, the KS GOF was used as a secondary 
method to test for normality after visually inspecting the Q-Q Plots. The results of the KS 
GOF are found in Appendix G, with highlighted areas being those MOPs with p-values 
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less than 0.05, which rejects the null hypothesis. The p-value is the smallest level of 
significance at which would be rejected when a specified test procedure is used on a 
given data set [Ref. 6:p. 334]. In general, the results of the test concur with the findings 
from the Q-Q Plots. 

The next assumption required for the two sample /-test is that both 
samples have a common variance. The test used to check the variance assumption was 
the F-test for common variances. The test determines if the ratio of the variances is equal 
to one. In this case, a two-sided F-test is used with a (X=0.05 level of significance. The 
hypothesis tested is 


^ (J JanusMOP ^ 
il . - - . __ 1 

o 2 ’ 

G JLinkMOP 


yy G JanusMOP 

versus, H ^\— - - 

G JLinkMOP 




The results of the F-test are in Appendix H, with the highlighted portions being those 
MOPs which rejected the null hypothesis at the a=0.05 level of significance and did not 
satisfy the common variance assumption. 

For MOPs detection range, kill range, and rounds fired, the /-test was 
applied regardless of the status of the assumptions in order to provide confidence 
intervals for further analysis. The resulting confidence intervals and p-values are found in 
Appendix I. 
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c. 


Power of the Two Sample t-Test 


Power is defined as the probability of rejecting a false null hypothesis 
[Ref. 8:p. 79]. The power of the t-test is the probability of detecting an acceptable 
difference A, in the sample means for each MOP. In terms of the null and alternative 
hypothesis, power is the probability of rejecting 

H„.A = 0, 

in favor of 


H^:A = predetermined value, 

for any true . 

The power of the two-sample t-test is a function of the sample sizes n and 
m, the significance level a, the population standard deviation a, and the real difference 
between the sample means A [Ref. 7 :p. 354, 394]. In this study, all scenarios were run an 
equal number of times; therefore the sample sizes were n = m = 10 for each MOP. The 
chosen significance level is a = 0.05. Using the assumption that the two populations 
from which the MOPs are drawn have a common standard deviation a, the pooled sample 
standard deviation is used to estimate a as follows: 

I 2 2 

/ S JanusMOP + S JUNKMOP , . , 

=,-, (since n = m). 


Determining the real A, or the value of the difference between means that 
is considered significant, is rather subjective. For FASCAM kills and chemical kills, a 
10% difference would generally be acceptable between the Janus stand-alone and JLink 
results. A total of 10 units were sent through the FASCAM minefields for all scenarios. 
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resulting in a value of A = 1 for FASCAM. A total of 20 units were sent through the 
chemical artillery for scenarios 1 and 2, and 200 units (due to aggregation of infantry) 
were sent through chemical artillery for scenario 3. This results in a value of A = 2 for 
chemical artillery in scenarios 1 and 2, and A = 20 for scenario 3. The accepted 
difference in mean LERs is 0.3. This difference is based on the expected attrition of each 
scenario and the impact such a difference would make on the outcome of the battle. The 
accepted difference in the mean rounds fired is approximately 20% of the total Janus 
rounds fired for each scenario. 

The accepted difference in mean detection ranges and kill ranges is 5% of 
the predominant weapon’s capabilities. The predominant weapon is defined as the most 
widely used weapon in the scenario, in this case the tank for scenarios 1 and 2, and the 
rifle for scenario 3. The maximum detection range for the tank and rifle is six and two 
kilometers respectively, resulting in a A = 0.3 for scenarios 1 and 2, and a A = 0.1 for 
scenario 3. The maximum kill range for the tank is four kilometers, while the rifle has a 
kill range of one kilometer. This results in a A = 0.2 for killing ranges in scenarios 1 and 
2, and A = 0.05 for scenario 3. The rationale for selecting 5% for the killing range 
follows. Assuming the tank travels at 20 kilometers per hour, then a 5% difference in 
killing range results in an additional killing range of 0.2 kilometers, or additional standoff 
time of 0.6 minutes. Since a tank is able to fire four rounds per minute, an acceptable 
difference of two additional rounds will be fired by either the Janus tank or JLink tank. 

The next step in determining the power is to calculate <}), the non-centrality 
parameter as follows: 
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where <7- is the standard deviation of the sample estimator for A [Ref. 7; p. 355, 394]. 
Further, 



This equation illustrates that as sample size increases, <p also increases, and likewise the 
power of the test. 


Appendix D shows the values required to calculate power. Table 2 shows 


the calculated power values for each MOP by scenario and environment. The power is 
determined by entering the noncentral T chart at the appropriate <})-value, using 18 degrees 
of freedom (since two means were estimated from 20 total data points), and reading the 
corresponding power value [Ref. 7:p. 355]. So, for example, our sample of size 10 ought 
to detect a 20% difference in “rounds fired” about 54% of the time (Table 2, Scenario #1, 
SWA). The results in Table 2 show that the experiment produces power values greater 
than 50% of detecting a difference A in the sample means for most MOPs. Also, power 
values for detection ranges and kill ranges are approximately 70% or greater in most 
scenarios. Power values in the ranges noted above are generally sufficient for analysis of 
combat simulations. 
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Scenario#!, SWA 


FASCAM 

Chem Kills 

LER 

DetRng 

KiURng 

Rnds 

Power 

1.00 

1.00 

.72 

1.00 

.88 


Scenario #1,HL 


FASCAM 

Qiem Kills 

LER 

DetRng 

KiURng 

Rnds 

Power 

.31 

.81 

.54 

1.00 

.93 

.29 

Scenario #2, SWA 


FASCAM 

Chem Kills 

LER 

DetRng 

KiURng 

Rnds 

Power 

.28 

1.00 

.83 

1.00 

.56 

.25 

Scenario #2, HL 


FASCAM 

Chem Kills 

LER 

DetRng 

KiURng 

Rnds 

Power 

.40 

.88 

.62 

1.00 

.94 

.40 

Scaiario#3,SWA 


FASCAM 

Chem Kills 

LER 

DetRng 

KiURng 

Rnds 

Power 

.40 

1.00 

.40 

1.00 

.69 

1.00 

Scenario #3, HL 


FASCAM 

Chem Kills 

LER 

DetRng 

KiURng 

Rnds 

Power 

.72 

1.00 

<20 

1.00 

.31 

.986 


Table 2. Power Values for MOPs 


3. Wilcoxon Test 

To substantiate the results of the two sample t-test and to test those MOPs which 
did not satisfy the /-test assumptions, the less powerful Mann-Whitney-Wilcoxon 
(Wilcoxon) nonparametric test was applied to compare the sample means. The Wilcoxon 
test uses the same two-tailed hypothesis test as the two sample /-test. The nonparametric 
test does not require that the data satisfy assumptions of normality and common variance. 
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The Wilcoxon test begins by rank-ordering all samples from both populations and 
assigning ranks 1 through m+n (20 in this case) to all values. If sample values are exactly 
equal, each value is assigned the average of the ranks that would have been assigned had 
there been no tie. The test statistic is found by first finding the sum (S) of the ranks 
assigned to the observations from the first population. The test statistic is given by 

T-S n{n + \) 

2 

The test statistic is then compared to the critical value for a two-sided test. The null 
hypothesis is rejected if 

7’<(9„/2or 

where ( 0 ^i 2 is the a/2quantile of T [Ref. 9:p. 224-226]. Conover [Ref. 9: Table 8] 

provides a table of critical values for various quantiles of T. The confidence level a is 
determined using the same Bonferroni method applied in the r-test. As with the /-test, the 
confidence level used was 

/ 6 = 0.0083. 

C. ANALYSIS OF MOPs 

The data set used to analyze FASCAM kills, chemical kills, and LERs is found in 
Appendix A. The MOPs detection range, kill range, and rounds fired were analyzed by 
side (Appendix B) to provide a clearer interpretation of the results and additional 
information to whether JLink is accurately simulating units performing a specific type of 
mission, in this case attacking and defending. 
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As previously mentioned, the methodology of analyzing the MOPs began with the 
ANOVA test. The ANOVA considers all three factors in the experiment at all levels to 
determine if the sample mean of a particular MOP in Janus is statistically similar to the 
sample mean of the same MOP in JLink. The results of the ANOVA tests in Appendix E 
conclude that significant differences exist in the sample means between the two modes, 
except in the instance of red rounds fired. These results indicate that the multiple factors 
in the experiment cause sample means to be different when analyzed as a whole. The 
reasons for the differences are initially apparent after inspecting the raw data in 
Appendices A and B. Specifically, Janus and JLink produced significantly different 
chemical kills for every scenario, and detection ranges and kill ranges are consistently 
larger for JLink. Given these disparate inputs to the ANOVA test, one would expect the 
test to generate statistically different sample means between the two modes. To provide 
further insight to the particular differences between Janus and its distributed counterpart, 
the analysis proceeded by identifying significant interactions between factors for each 
MOP and applying the pair-wise two sample r-test and Wilcoxon test to determine where 
differences existed, leading to recommendations for improvements to JLink. 

1. FASCAM 

FASCAM was tested using the results in Table 3. 
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Summary Statistics 

Janus Stand- 

JLink vs 



Alone 

JLink 

Armored SWA 

Average 

2.7 

2 

Variance 

2.68 

1.33 


Stand Dev 

1.63 

1.15 

Armored HL 

Average 

2.9 

1.5 

Variance 

2.99 

.72 


Stand Dev 

1.73 

.85 

Armored Coalition SWA 

Average 

2.3 

1.5 

Variance 

2.68 

1.61 


Stand Dev 

1.63 

1.27 

Armored Coalition HL 

Average 

2.6 

.6 

Variance 

2.49 

.27 


Stand Dev 

1.58 

.52 

Light Infantry SWA 

Average 

Variance 

1.1 

2.54 

.2 

.4 


Stand Dev 

1.60 

.63 

Light Infantry HL 

Average 

Variance 

1 

1.11 

.6 

.27 


Stand Dev 

1.05 

.52 


Table 3. FASCAM Results 


First, the Ntest assumptions were examined. Table 4 summarizes the results of 
visually inspecting the Q-Q Plots in Appendix F and the results of the KS GOF test in 
Appendix G. The highlighted portions are those areas which did not satisfy the normal 
assumption. 



Scenario#!, SWA 

Scenario #1,HL 

Scenario #2, SWA 

Mode 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

Normal 

yes 

yes 

yes 

yes 

yes 

yes 



Scenario #2, HL 

Scenario #3, SWA 

Scenario #3, HL 

Mode 

Janus 

JLink 

Janus 

JLink 

Janus 

Jlink 

Normal 

no 

no 

yes 

no 

yes 

no 


Table 4. FASCAM Normality 


Next, the F-test was used to check for common variance between the Janus and 
JLink populations. The results of the F-tests are in Appendix H. Table 5 summarizes the 
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results of the tests for normality and common variance for each scenario. The highlighted 
portions are those populations which failed either the normality assumption or the 
common variance assumption, or both. The two sample t-test was only performed on 
those Janus/JLink pairs that met the normality assumption and passed the common 
variance test. 



Scenario#!, SWA 

Scenario #1,HL 

Scenario #2, SWA 

Mode 

Janus 

JUnk 

Janus 

JLink 

Janus 

JLink 

Assumptions 

yes 

no 

yes 



Scenario #2, HL 

Scenario #3, SWA 

Scenario #3, HL 

Mode 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

Assumptions 

no 

no 

no 


Table 5. FASCAM Satisfying Normality and Common Variance 
Appendix I provides results of the pair-wise two sample r-tests, to include p- 
values and confidence intervals (Cl), and highlights those pairs which reject the null 
hypothesis. Table 6 summarizes the FASCAM results. Note that both pairs tested failed 
to reject the null hypothesis. 



Scenario#!, SWA 

Scenario#!, HL 

Scenario #2, SWA 

Mode 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

Results 

M tone] null hyp 

na 

fail to lej null hyp 



Scenario #2, HL 

Scenario #3, SWA 

Scenario #3, HL 

Mode 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

Results 

na 

na 

na 


Table 6. FASCAM Results of Two Sample t-Test 
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The Wilcoxon test was then applied to all pair-wise comparisons of 
FASCAM. Appendix J provides the results of the Wilcoxon test, and highlights those 
pairs which reject the null hypothesis. Tables 7 summarizes the FASCAM results and 
highlights those pairs which reject the null hypothesis that the two sample averages are 
statistically similar. 



Scenario#!, SWA 

Sceiiario#l,HL 

Scenario #2, SWA 

Mode 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

Results 

Ml to lej null hyp 

fail to lej null hyp 

fail to rej null hyp 



Scenario #2, HL 

Scenario #3, SWA 

Scenario #3, HL 

Mode 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

Results 

ngmiBbyp^ 

fail to lej null hyp 

fail to rej null hyp 


Table 7. FASCAM Results of Wilcoxon Test 


2. Chemical Kills 

The chemical kills MOP was analyzed using the results in Table 8. 
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Summary Statistics 

Janus Stand- 

JLink vs 



Alone 

JLink 

Armored SWA 

Average 

4.4 

.3 

Variance 

.49 

.23 


Stand Dev 

.70 

.48 

Armored HL 

Average 

12.3 

2.5 

Variance 

3.12 

1.39 


Stand Dev 

1.77 

1.18 

Armored Coalition SWA 

Average 

3.8 

.1 


Variance 

.4 

.1 


Stand Dev 

.63 

.32 

Armored Coalition HL 

Average 

12.3 

2.9 

Variance 

2.68 

.99 


Stand Dev 

1.64 

.99 

Light Infantry SWA 

Average 

Variance 

9.9 

10.1 

.6 

1.6 


Stand Dev 

3.18 

1.26 

Light Infantry HL 

Average 

Variance 

69.6 

1.6 

36.6 

77.6 


Stand Dev 

1.26 

8.81 


Table 8. Chemical Results 


The analysis applied the two sample t-test to identify the specific scenarios which 
produced statistically different chemical kills between Janus and JLink. First, the 
assumptions were checked in the same manner as with FASCAM. Normality was 
determined based on the visual inspection of the Q-Q Plots and the results of the KS GOF 
test. Then, common variance was determined based on the results of the F-test. Table 9 
summarizes the results of the tests for normality and common variance for each scenario. 
The highlighted portions are those populations which failed either the normality 
assumption or the common variance assumption, or both. Again, the two sample r-test 
was only performed on those Janus/JLink pairs that met the normality assumption and 
passed the common variance test. 
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Scenario#!, SWA 

Scenario #1,HL 

Scenario #2, SWA 

Mode 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

Assumptions 

no 

yes 

no 



Scenario #2, HL 

Scenario #3, SWA 

Scenario#3,HL 

Mode 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

Assumptions 

no 

no 

no 


Table 9. Chemical Kills Satisfying Normality and Common Variance 

The r-test results in Appendix I show that the one pair-wise comparison tested 
rejected the null hypothesis that the average number of chemical kills between Janus and 
JLink are statistically similar in scenario #1, HL. The result is summarized (for 
consistency) in Table 10. 



Scenario#!, SWA 

Scenario #!,HL 

Scenario #2, SWA 

Mode 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

Results 

na 

ig null hyp 

na 



Scenario #2, HL 

Scenario #3, SWA 

Scenario #3, HL 

Mode 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

Results 

na 

na 

na 


Table 10. Chemical Results of Two Sample t-Test 


The Wilcoxon test was then applied to all pair-wise comparisons of chemical 
kills. Appendix J provides the results of the Wilcoxon test, and highlights those pairs 
which reject the null hypothesis. Tables 11 summarizes the chemical results and 
highlight those pairs which reject the null hypothesis that the two sample averages are 
statistically similar. 
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Scenario#!, SWA 

Scenario #1,HL 

Scenario #2, SWA 

Mode 

Janus 

JUnk 

Janus 

JUnk 

Janus 

JUnk 

Results 

igniiBhyp 

ig null hyp 

pgnuUhyp, 



Scenario #2, HL 

Scenario #3, SWA 

Scenario#3,HL 

Mode 

Janus 

JUnk 

Janus 

JUnk 

Janus 

JUnk 

Results 

ig null hyp 

rej null hyp 

ignuRhyp 


Table 11. Chemical Results of Wilcoxon Test 


3. Detection Range (Blue/Red) 


The detection range MOP was analyzed using the results in Tables 12 and 13. As 


previously mentioned, the data for detection range was analyzed separately by blue and 


red force. 



Summary Statistics 

Janus Stand- 

JLink vs 



Alone 

JLink 

Armored SWA 

Average 

3.689 

4.171 

Variance 

.010 

.005 


Stand Dev 

.098 

.072 

Armored HL 

Average 

3.330 

3.951 

Variance 

.005 

.012 


Stand Dev 

.072 

.107 

Armored Coalition SWA 

Average 

3.619 

4.073 

Variance 

.033 

.011 


Stand Dev 

.181 

.104 

Armored Coalition HL 

Average 

3.362 

3.948 

Variance 

.009 

.006 


Stand Dev 

.096 

.074 

Light Infantry SWA 

Average 

Variance 

2.067 

.002 

2.075 

.001 


Stand Dev 

.040 

.027 

Light Infantry HL 

Average 

Variance 

1.648 

.001 

1.719 

.001 


Stand Dev 

.033 

.036 


Table 12. Blue Detection Range Results 
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Summary Statistics 

Janus Stand- 

JLink vs 



Alone 

JLink 

Armored SWA 

Average 

2.470 

3.389 

Variance 

.027 

.021 


Stand Dev 

.165 

.146 

Armored HL 

Average 

2.000 

2.882 

Variance 

.012 

.034 


Stand Dev 

.108 

.183 

Armored Coalition SWA 

Average 

Variance 

2.212 

.011 

3.242 

.027 


Stand Dev 

.103 

.163 

Armored Coalition HL 

Average 

1.888 

2.624 

Variance 

.007 

.054 


Stand Dev 

.081 

.232 

Light Infantry SWA 

Average 

Variance 

1.791 

.001 

2.008 

.001 


Stand Dev 

.030 

.030 

Light Infantry HL 

Average 

Variance 

1.559 

.001 

1.672 

.002 


Stand Dev 

.027 

.041 


Table 13. Red Detection Range Results 


The ANOVA interaction plots in Appendix E for both blue and red detections show 
significant interaction between the scenario and the mode, illustrated by the larger gap in 
detection differences for scenarios 1 and 2 as compared to the detection difference for 
scenario 3. Such interaction is expected given that scenarios 1 and 2 used weapon 
systems with detection capability up to six kilometers, whereas scenario 3 used infantry 
units with detection capability of roughly two kilometers. 

Testing for t-test assumptions showed that all detection populations satisfied the 
normality assumption and only the red detection range in scenario # 2, HL rejected the 
assumption of common variance. Tables 14 and 15 show the summarized results of the t- 
test for blue and red detection ranges, respectively. 
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Scenario#!, SWA 

Scenario#l,HL ^ 

Scenario #2, SWA 

Mode 

Janus 

JUnk 

Janus 

JUnk 

Janus 

JUnk 

Results 

Kgnullh}p 

ngnullh>p 

tgiHiUhyp 



Scenario #2, HL 

Scenario #3, SWA 

Scenario#3,HL 

Mode 

Janus 

JUnk 

Janus 

JUnk 

Janus 

JUnk 

Results 

rig null h)p 

fail to ig null hyp 

igrHiIlhyp 


Table 14. Blue Detection Range Results of t-Test 



Scenario#!, SWA 

Scenario #!,HL 

Scenario #2, SWA 

Mode 

Janus 

JUnk 

Janus 

JUnk 

Janus 

JUnk 

Results 

rg nun hyp 

ig null hyp 

lejnunhyp 



Scenario #2, HL 

Scenario #3, SWA 

Scenario #3, HL 

Mode 

Janus 

JUnk 

Janus 

JUnk 

Janus 

JUnk 

Results 

na 

ig null hyp 

ig null hyp 


Table 15. Red Detection Range Results of /-Test 


The Wilcoxon test was also applied to all pair-wise comparisons of detection 
ranges. Tables 16 and 17 summarize the results from Appendix J and highlight those 
pairs which reject the null hypothesis that the two sample averages are statistically 
similar. 
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Scenario#!, SWA 

Scenario#!, HL 

Scenario #2, SWA 

Mode 

Janus 

JLink 

Janus 

JLink 

Janas 

JLink 

Results 

ig null hyp 

rg null hyp 

ig null hyp 



Scenario #2, HL 

Scenario #3, SWA 

Scenario #3, HL 

Mode 

Janus 

JLink 

Janus 

JLink 

Janas 

JLink 

Results 

rgnuDhyp 

fail to rej null hyp 

ignulil^ 


Table 16. Blue Detection Range Results of Wilcoxon Test 



Scenario #!, SWA 

Scenario #!,HL 

Scenario #2, SWA 

Mode 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

Results 

rgnulll^ 

ig null hyp 

ig rain hyp 



Scenario #2, HL 

Scenario #3, SWA 

Scenario #3, HL 

Mode 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

Results 

ig nun hyp 

ig null hyp 

ig null hyp 


Table 17. Red Detection Range Results of Wilcoxon Test 


4. Kill Ranges (Blue/Red) 

The kill range MOP was analyzed using the data in Tables 19 and 20. The data 
for kill ranges was analyzed separately by blue and red force. 
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Summary Statistics 

Janus Stand- 

JLink vs 

- 


Alone 

JLink 

Armored SWA 

Average 

2.074 

2.394 

Variance 

.123 

.006 


Stand Dev 

.350 

.076 

Armored HL 

Average 

1.955 

2.556 

Variance 

.045 

.061 


Stand Dev 

.212 

.246 

Armored Coalition SWA 

Average 

1.987 

2.325 

Variance 

.136 

.216 


Stand Dev 

.369 

.465 

Armored Coalition HL 

Average 

1.831 

2.145 

Variance 

.050 

.120 


Stand Dev 

.224 

.346 

Light Infantry SWA 

Average 

Variance 

1.477 

.005 

1.626 

.001 


Stand Dev 

.072 

.027 

Light Infantry HL 

Average 

Variance 

1.301 

.003 

1.375 

.001 


Stand Dev 

.055 

.023 


Table 18. Blue Kill Range Results 



Summary Statistics 

Janus Stand- 

JLink vs 



Alone 

JLink 

Armored SWA 

Average 

2.847 

3.100 

Variance 

.033 

.061 


Stand Dev 

.181 

.246 

Armored HL 

Average 

2.607 

3.00 

Variance 

.016 

.007 


Stand Dev 

.127 

.084 

Armored Coalition SWA 

Average 

3.00 

3.00 

Variance 

.025 

.065 


Stand Dev 

.158 

.255 

Armored Coalition HL 

Average 

2.626 

3.012 

Variance 

.018 

.009 


Stand Dev 

.135 

.095 

Light Infantry SWA 

Average 

Variance 

1.313 

.005 

1.369 

.004 


Stand Dev 

.067 

.067 

Light Infantry HL 

Average 

Variance 

1.317 

.004 

1.455 

.018 


Stand Dev 

.067 

.133 


Table 19. Red Kill Range Results 

The ANOVA interaction plots illustrate a significant interaction between the environment 
and the mode for red kill ranges. This interaction is interpreted as, not only are the mean 
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red kill ranges different between Janus and JLink, the differences are greater in HL terrain 
than SWA terrain. 

Testing for r-test assumptions showed that all red kill range populations satisfied 
both the normality and common variance assumptions. Table 20 summarizes which 
scenarios satisfied the assumptions for blue kill ranges. 



Scenario#!, SWA 

Scenario #1,HL 

Scenario #2, SWA 

Mode 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

Assumptions 

no 

yes 

no 



Scenario #2, HL 

Scenario #3, SWA 

Scenario #3, HL 

Mode 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

Assumptions 

yes 

DO 

s.- 

no 


Table 20. Blue Kill Ranges Satisfying Normality and Common Variance 

Tables 21 and 22 summarize the results from Appendix I of the two sample r-test 
for blue and red kill ranges, respectively. 



Scenario#!, SWA 

Scenario #!,HL 

Scenario #2, SWA 

Mode 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

Results 

na 


na 



Scenario #2, HL 

Scenario #3, SWA 

Scenario #3, HL 

Mode 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

Results 

fail to rcj null hyp 

na 

na 


Table 21. Blue Kill Range Results of t-Test 
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Scenario #1, SVVA 

Scenario #1,HL 

Scenario #2, SWA 

Mode 

Janus 

JUnk 

Janus 

JUnk 

Janus 

JUnk 

Results 

fail to rej null hyp 

null hyp 

fail to lej null hyp 



Scenario #2, HL 

Scenario #3, SWA 

Scenario #3, HL 

Mode 

Janus 

JUnk 

Janus 

JUnk 

Janus 

JUnk 

Results 

rg nun hyp 

fail to iq null hyp 

fail to iq null hyp 


Table 22. Red Kill Range Results of t-Test 


The Wilcoxon test was then applied to all pair-wise comparisons of kill ranges. 
Tables 23 and 24 summarize the results from Appendix J and highlight those pairs which 
reject the null hypothesis that the two sample averages are statistically similar. 



Scenario#!, SWA 

Scenario #1,HL 

Scenario #2, SWA 

Mode 

Janus 

JUnk 

Janus 

JUnk 

Janus 

JUnk 

Results 

fail to lej null hyp 

iq null hyp 

foil to iq null hyp 



Scenario #2, HL 

Scenario #3, SWA 

Scenario #3, HL 

Mode 

Janus 

JUnk 

Janus 

JUnk 

Janus 

JUnk 

Results 

fail tore] null hyp 

iq nun hyp 

iq null hyp 


Table 23. Blue Kill Range Results of Wilcoxon Test 
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Scenario #1» SWA 

Scenario #1,HL 

Scenario #2, SWA 

Mcxie 

Janus 

JUnk 

Janus 

JUnk 

Janus 

JUnk 

Results 

fail to rej null hyp 

ig null hyp 

fail to rej null hyp 



Scenario #2, HL 

Scenario #3, SWA 

Scenario#3,HL 

Mode 

Janus 

JUnk 

Janus 

JUnk 

Janus 

JUnk 

Results 

ig null hyp 

fail to rej null hyp 

fail to rej null hyp 


Table 24. Red Kill Range Results of Wilcoxon Test 


5. Rounds Fired (Blue/Red) 

The rounds fired MOP was analyzed using the data in Tables 25 and 26. The data 
for rounds fired was analyzed separately by blue and red force. 



Summary Statistics 

Janus Stand- 
Alone 

JLink vs 
JLink 

Armored SWA 

Average 

Variance 

Stand Dev 

85.3 

1292.23 

35.95 

32.8 

101.73 

10.09 

Armored HL 

Average 

Variance 

Stand Dev 

112.3 

1360.01 

36.88 

58.1 

40.32 

6.35 

Armored Coalition SWA 

Average 

Variance 

Stand Dev 

56.5 

176.5 

13.29 

35.3 

206.01 

14.35 

Armored Coalition HL 

Average 

Variance 

Stand Dev 

71.6 

401.6 

20.04 

73.6 

424.04 

20.59 

Light Infantry SWA 

Average 

Variance 

Stand Dev 

969.6 

63081.38 

251.16 

621.3 

1896.46 

43.55 

Light Infantry HL 

Average 

Variance 

Stand Dev 

1347.6 

93197.38 

305.28 

1341.5 

31205.83 

176.65 


Table 25. Blue Rounds Fired Results 
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Summary Statistics 

Janus Stand- 

JLink vs 



Alone 

JLink 

Armored SWA 

Average 

110.1 

195.3 

Variance 

1215.43 

983.79 


Stand Dev 

34.86 

31.37 

Armored HL 

Average 

531.9 

323.8 

Variance 

75729.21 

6535.07 


Stand Dev 

275.19 

80.84 

Armored Coalition SWA 

Average 

144 

165.7 

Variance 

6945.78 

918.46 


Stand Dev 

83.34 

30.31 

Armored Coalition HL 

Average 

523.4 

411.9 

Variance 

19137.16 

18999.88 


Stand Dev 

138.34 

137.84 

Light Infantry SWA 

Average 

Variance 

1626 

11621.11 

1619 

4510 


Stand Dev 

107.80 

67.16 

Light Infantry HL 

Average 

Variance 

1365.5 

64852.5 

1594 

53604.44 


Stand Dev 

254.66 

231.53 


Table 26. Red Rounds Fired Results 


The results of the ANOVA test in Appendix E show that the average red rounds fired in 
JLink is statistically similar to the average red rounds fired in Janus, as evidenced by the 
p-value of .96 for Mode. Because of this result, no further statistical tests were conducted 
on the red rounds fired. Also, the results of the ANOVA interaction plots illustrate 
interaction between the environment and mode for blue rounds fired. The difference 
between Janus and JLink in the mean blue rounds fired in SWA is significantly larger 
than the mean blue rounds fired in HL. 

Next, the two sample t-test was applied to all pairs of blue rounds fired. Results 
from testing for normality and common variance are summarized in Table 27. 
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Scenario#!, SWA 

Scenario #1,HL 

Scenario #2, SWA 

Mode 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

Assumptions 

no 

no 

no 



Scenario #2, HL 

Scenario #3, SWA 

Scenario #3, HL 

Mode 

Janus 

JUnk 

Janus 

JLink 

Janus 

JLink 

Assumptions 

yes 

no 

yes 


Table 27. Blue Rounds Fired Satisfying Normality and Common Variance 

Table 28 summarizes the results from Appendix I of the two sample t-test for blue 
rounds fired. 



Scenario#!, SWA 

Scenario #1,HL 

Scenario #2, SWA 

Mode 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

Results 

na 

na 

na 



Scenario #2, HL 

Scenario #3, SWA 

Scenario #3, HL 

Mode 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

Results 

fail tone] null hyp 

na 

fail to rej null hyp 


Table 28. Blue Rounds Fired Results of t-Test 


The Wilcoxon test was then applied to all pair-wise comparisons of blue rounds 
fired. Table 29 summarizes the results from Appendix J and highlights those pairs which 
reject the null hypothesis that the two sample averages are statistically similar. 
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Scenario#!, SWA 

Scenario #1,HL 

Scenario #2, SWA 

Mode 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

Results 

ig null hyp 

ng null hyp 

fail to rej null hyp 



Scenario #2, HL 

Scenario #3, SWA 

Scenario #3, HL 

Mode 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

Results 

fail to rej null hyp 

ng null hyp 

fail to rej null hyp 


Table 29. Blue Rounds Fired Results of Wilcoxon Test 


6. Loss Exchange Ratio (LER) 

The LER MOP was analyzed using the results in Table 30. 



Summary Statistics 

Janus Stand- 
Alone 

JLink vs 
JLink 

Armored SWA 

Average 

Variance 

Stand Dev 

2.182 

.072 

.269 

1.161 

.074 

.273 

Armored HL 

Average 

Variance 

Stand Dev 

1.659 

.156 

.395 

1.505 

.049 

.221 

Armored Coalition SWA 

Average 

Variance 

Stand Dev 

1.377 

.025 

.160 

1.174 

.070 

.265 

Armored Coalition HL- 

Average 

Variance 

Stand Dev 

1.489 

.040 

.201 

1.835 

.122 

.350 

Light Infantry SWA 

Average 

Variance 

Stand Dev 

1.557 

.254 

.504 

.937 

.011 

.104 

Light Infantry HL 

Average 

Variance 

Stand Dev 

3.112 

1.004 

1.002 

2.36 

.436 

.660 


Table 30. LER Results 


The analysis applied the two sample r-test to identify the specific scenarios which 


produced statistically different LERs between Janus and JLink. First, the r-test 


assumptions were checked in the same manner as before. The tests revealed that all 
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scenarios satisfied both the normality and common variance assumption except scenario 


#3, SWA. The results of the t-tests are summarized in Table 31. 



Scenario#!, SWA 

Scenario #1,HL 

Scenario #2, SWA 

Mode 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

Results 

rg null hyp 

fail to iq null hyp 

fail to rej null hyp 



Scenario #2, HL 

Scenario #3, SWA 

Scenario #3, HL 

Mode 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

Results 

fail to rg null hyp 

na 

fail to lej null hyp 


Table 31. LER Results of Two Sample t-Test 


The Wilcoxon test was then applied to all pair-wise comparisons of LERs. Tables 
32 summarizes the LER test results and highlights those pairs which reject the null 
hypothesis that the two sample averages are statistically similar. 



Scenario#!, SWA 

Scenario #!,HL 

Scenario #2, SWA 

Mode 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

Results 

ig null hyp 

fail to rej null hyp 

fail tong null hyp 



Scenario #2, HL 

Scenario #3, SWA 

Scenario #3, HL 

Mode 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

Results 

fail to lej null hyp 

ignuDhyp 

fail to rej null hyp 


Table 32. LER Results of Wilcoxon Test 


D. VARIABILITY WITHIN JLINK 

The data in Appendix C are the results of 10 JLink runs, using the same scenarios 
and environments as described before, except that the same random number seed was 
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used for all runs. The reason for conducting these runs was to determine the variance 
within the JLink system and compare that variance to runs using different random number 
seeds. In essence, this test isolated the variance contributed by JLink alone. 

Inspecting the variances in Appendix C, and comparing them to the variances of 
the runs using 10 different random number seeds in Appendix A, we see that the JLink 
variance using the same seed was generally slightly smaller than the variance in Appendix 
A. This finding indicates that most of the variability is generated within the runs, instead 
of from the changes in the random number seed. 
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V. ANALYSIS OF RESULTS 


The results of the statistical tests from Chapter IV provide a firm basis for 
assessing the differences between the JLink system and its parent simulation, Janus. This 
chapter will analyze the results of the ANOVA test, two sample /-test, and Wilcoxon test 
and offer insights to the causes of these differences. Since the ANOVA test rejected the 
null hypothesis that the averages were statistically similar between Janus and JLink for all 
MOPs except red rounds fired, the analysis will focus primarily on where the differences 
occurred based on the results of the /-test and Wilcoxon test. The analysis includes 
possible causes for unexpected results and provides recommendations to resolve those 
issues derived from such results. The analysis of the results is presented by individual 
MOP. 


A. FASCAM KILLS 

The results from the two sample /-test and Wilcoxon test rejected the null 
hypothesis that the average FASCAM kills are the same between Janus stand-alone and 
JLink in only one instance, for scenario #2, HL. This would lead one to believe that 
JLink is accurately representing FASCAM under various conditions. Also, four out of six 
times Jlink produced average results which were within the accepted tolerance of A = 1 
kill. However, two issues must be addressed before drawing a final conclusion about 
JLink FASCAM: the location of the FASCAM minefield and the trend in the data 
between Janus and JLink. 
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As mentioned in Chapter III, paragraph C. 2, the player fires FASCAM from an 
artillery platform after the scenario begins. To remove aim-error variability due to human 
interaction, all runs were “pucked” to provide a common firing point between scenarios. 
Ballistic errors in azimuth and elevation are another source of location variability when 
firing from an artillery platform. Janus simulates these ballistic errors by drawing the x 
and y coordinates of the round’s impact point from a normal distribution. So, in order to 
ensure that a FASCAM minefield is placed precisely in the same location in different 
runs, the runs must be “pucked” under almost identical conditions to ensure reasonable 
azimuth and elevation errors. The experiment satisfied the first condition, but due to 
JLink architecture and different random number generators, dispersion caused by the 
latter condition could not be completely controlled. 

To reduce the dispersion of the minefields, FASCAM was fired in a 
relatively small geographic location, compared to the entire scenario. By firing five 
minefields per run, for 10 runs, in a relatively small location, one would expect the effect 
of minefield dispersion to be negligible over the entire experiment. However, after 
replaying some of the scenarios on the Janus Analyst Workstation and comparing the 
locations of the minefields between modes, it was evident that minefield dispersion 
would contribute to dissimilar FASCAM kills in Janus and JLink. Recall that blue forces 
fired FASCAM at specified red units traveling on a narrow route. Suppose that four of 
the five FASCAM minefields fired in Janus landed on the designated route whereas only 
three JLink minefields landed on the route, due to ballistic error. Under these 
circumstances, the experiment would not isolating the effects of the FASCAM 
minefields. 
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Another issue regarding the FASCAM results is the consistent trend towards 
greater FASCAM kills in Janus as compared to JLink. Such a trend forces the analyst to 
proceed with caution when drawing conclusions from the statistical tests. Although the 
tests indicate that the FASCAM results are statistically similar between modes (except in 
one instance), such a trend may indicate that the JLink FASCAM does not produce the 
same number of kills per mine as Janus FASCAM. 

Given the results of the tests, the current configuration of JLink provides a fair 
representation of FASCAM as compared to Janus. Although there appears to be a trend 
in average FASCAM kills favoring Janus, the significance of the trend is diminished 
somewhat by the fact that one out of every six runs produced greater JLink FASCAM 
kills than Janus. The trend in average kills, however, does merit further investigation. 
Additional testing, isolating the effects of FASCAM, is necessary to determine if the 
JLink FASCAM is as lethal as Janus FASCAM. 

B. CHEMICAL KILLS 

In all cases, the r-test and the Wilcoxon test rejected the null hypothesis 
that the average chemical kills are the same between Janus stand-alone and JLink. 
Clearly, JLink’s current configuration does not accurately replicate the effects of Janus 
chemical artillery. 

In an attempt to identify the cause of the disparate test results and rectify 
the shortcoming, the JLink delivery method shown in Figure 2 was investigated. As 
aforementioned, Janus calculates artillery by volley as opposed to individual rounds, as 
dictated by the DIS standard. Calculating the artillery by volley results in a single large 


65 


chemical cloud which stays together for a much longer period of time as opposed to 
individual clouds which dissipate quickly and cover less area. Since the smaller clouds 
dissipate quickly, entities in contact are affected less by the smaller chemical clouds than 
by the larger aggregated cloud. 

In order to correct this shortcoming, the World Modeler software was rewritten so 
that whenever a user participates in a JLink to JLink game, all chemical artillery will be 
fired and portrayed by volley as opposed to individual rounds. After making the 
corrections in the code, 10 additional runs were performed using the SWA armored 
scenario. The results of the scenario were more in line with what was originally 
expected. 

The ten runs of both Janus and JLink yielded an average of 10.3 and 10.5 
chemical kills respectively (Table 33). Because the data satisfied the assumptions of 
normality and common variance with a Q-Q Plot and F-test respectively, a two-sample t- 
test was also conducted. Since the generated test statistic 0.7356 is less than the critical 
value of 2.101, the conclusion is that the means do not differ significantly. 


RUN# 

Chemical Kills 

Janus 

JLink 

1 

9 

9 

2 

11 

11 

3 

12 

11 

4 

9 

11 

5 

9 

11 

6 

11 

12 

7 

11 

10 

8 

8 

11 

9 

11 

8 

10 

12 

11 

XBAR 

10.3 

10.5 

VAR 

2.011 

1.389 

STDE 

1.418 

1.179 


Table 33. Chemical Kills for Volley Fire 
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These results support the finding that JLink accurately represents chemical 
artillery in a SWA armored scenario. Additional testing using other scenarios is 
necessary to conclude that JLink accurately portrays chemical artillery under varied 
conditions. The results show that chemical rounds must be fired in volleys, as opposed to 
individual rounds, whenever a user plays a JLink to JLink game. Software upgrades will 
include a toggle transparent to the user so that JLink automatically sends out volleys as 
opposed to individual rounds whenever a user indicates that the other participating DIS 
simulation is another JLink. 

C. DETECTION RANGES (BLUE/RED) 

The next MOP analyzed was detection range, analyzed separately by blue force 
and red force, using the data from Appendix B. (The headings “Det Ranges Blue/Red” in 
Appendix B refer to the those units doing the detecting). As mentioned previously, 
separating the detection ranges by side offers a clearer interpretation of the results and 
additional information to whether JLink is accurately simulating units performing a 
specific type of mission, in this case attacking and defending. 

The two-sample r-test and the Wjlcoxon test rejected the null hypothesis that the 
average blue and red detection ranges are similar for every scenario except for the blue 
detection ranges in scenario #3, SWA. 

Next, the thesis compared the accepted A = 0.3 kilometer tolerance for detection 
ranges in scenarios 1 and 2, and A = 0.1 kilometers for scenario 3, to the confidence 
intervals from Appendix I, to determine if any detection ranges were within acceptable 
tolerance. The comparison concluded that none of the detection ranges were within the 
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acceptable tolerance, with the one exception noted above. (Figure 7 illustrates the 
method for applying the A value to the confidence interval for scenario #3, HL, blue 
detection range). Additionally, a consistent trend in the detection data surfaced; the JLink 
detection ranges were always larger than the Janus detection ranges, for both the blue and 
red forces. 



Figure 7. Confidence Intervals Versus Acceptance Range 

The next step was to identify the cause of the disparate results and the trend in the 
data. The first issue investigated was the passing of the entity’s z-coordinate. If the 
entity’s z-coordinate is not passed properly to JLink, or interpreted properly by JLink, the 
entity could be simulated above ground level, resulting in better line of sight and 
extended detection ranges. This hypothesis turned out to be incorrect, however, because 
JLink can determine which entities are ground entities. When the information about a 
ground entity is passed to JLink, JLink assigns the entity the x and y coordinates passed 
from Janus and places the entity on the ground. The ground entity then uses the ground z- 
coordinate to perform line of sight calculations. 
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The second proposed solution focused on the fact that something in the battlefield 
environment was affecting the detection ranges. The issue of artillery delivered smoke 
and munitions was suggested as a possible cause. All scenarios used large volumes of 
artillery smoke and artillery High Explosive (HE) rounds. As discussed in the analysis of 
chemical kills, Janus and JLink simulated artillery delivery methods differently in this 
experiment. If the dimension of the chemical cloud could affect the number of chemical 
kills, then the dimensions of any cloud generated from an artillery munition may also 
affect the range at which an entity can detect through the cloud. 

To test the proposed solution, the SWA armored scenario was re-run, excluding 
all artillery from both sides. Thus, in this case there were no clouds resulting from 
artillery smoke or HE rounds on the battlefield. The hypothesis was that the artillery 
simulated as a volley (Janus) led to decreased visibility, and shorter detection ranges, than 
artillery simulated as individual rounds (JLink). The scenario was then run 10 times in 
both the Janus and JLINK modes, using the same random number seeds. The results 
yielded an average detection range for red Janus entities of 2.532 kilometers, while the 
average detection range for the red JLINK entities was 2.463 kilometers. The average 
detection range for blue Janus entities was 3.885 versus 3.767 for the blue JLINK entities. 
Table 34 depicts the actual ranges for each run. The assumption of normality was 
satisfied, but the F-Test concluded that the variances for the populations were not the 
same for the blue Janus and blue JLink detection ranges. The variances were the same, 
however, for the red Janus and red JLink detection ranges. Finally, a two-sample Mest 
showed that in both cases, the means for the populations were not significantly different. 
This analysis provides a basis for concluding that the different artillery delivery methods 
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contributed to the disparate detection results. Further, this thesis concludes that the 


volley method used by Janus to deliver artillery munitions generates shorter detection 


ranges than the individual round method due to the dimensions and dissipation rate of the 


cloud. 


Currently, the software has not been rewritten in JLink to fire artillery using the 


same method as Janus. After the code is rewritten, several additional experiments will be 


done to ensure that the artillery is being accurately played using the volley method, and 


that this method generates similar detection ranges between Janus and JLink. 



Det Ranges Blue 

Det Ranges Red 

RUN# 

Janus 

JLink 

Janus 

JLink 

1 

3.863 

3.785 

2.294 

2.711 

2 

3.940 

3.753 

2.513 

2.386 

3 

3.846 

3.846 

2.541 

2.541 

4 

3.778 

3.877 

2.408 

2.719 

5 

3.918 

3.918 

2.890 

2.889 

6 

4.017 

3.616 

2.912 

2.166 

7 

3.863 

3.567 

2.294 

2.319 

8 

3.912 

4.021 

2.715 

2.512 

9 

3.856 

3.660 

2.377 

2.275 

10 

3.853 

3.631 

2.377 

2.119 

XBAR 

3.885 

3.767 

2.532 

2.464 

VAR 

0.004 

0.022 

0.054 

0.065 

STDEV 

0.065 

0.149 

0.232 

0.255 

f-Test Stat 

0.040 

.515 

Crit. Value 

2.101 

2.101 


Table 34. Detection Ranges Without Artillery 


D. KILL RANGES (BLUE/RED) 

The next MOP analyzed was kill range, analyzed separately by blue force and red 
force, using the data from Appendix B. (The headings “Kill Ranges Blue/Red” in 


Appendix B refer to the those units being killed). The results of the two-sample r-test and 
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the Wilcoxon test rejected the null hypothesis that the average kill ranges (blue and red) 
are same between modes in six instances : 

1. Blue Kill Ranges in Scenario #1, SWA 

2. Blue Kill Ranges in Scenario #1, HL 

3. Red Kill Ranges in Scenario #1, HL 

4. Red Kill Ranges in Scenario #2, HL 

5. Blue Kill Ranges in Scenario #3, SWA 

6. Blue Kill Ranges in Scenario #3, HL 

Next, the acceptable tolerance range of A = 0.2 kilometers for scenarios 1 and 2, 
and A = 0.05 kilometers for scenario 3, was compared to the kill range CIs in Appendix I 
for those scenarios which rejected the null hypothesis. The results of the comparison 
were that none of the CIs fell within the acceptable tolerance. This finding can be 
interpreted by stating that the mean kill ranges in Janus and JLink in these scenarios are 
not the same, and indeed so dissimilar that the difference in the means falls outside the 
acceptable tolerance. 

As with the detection ranges data, a consistent trend occurred in the kill range 
data; the kill ranges for JLink were consistently greater than the kill ranges for Janus. 

This result is not surprising in light of the similar trend in detection ranges. There is a 
strong likelihood that if an entity can detect at further range, then it can acquire and target 
at greater distances, and subsequently kill at longer ranges. This reasoning is based on 
the method by which a Janus entity performs the engagement sequence. After conducting 
a line of sight calculation, Janus determines the target’s location and identity. Once 
identified, Janus determines the target’s priority and places it on the target list. The target 
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list for each Janus entity determines which targets will be engaged and in what order. 
Janus then acquires the target and engages. The engagements may, or may not, lead to a 
kill. This entire sequence, however, stems from the initial detection. Therefore, if the 
detection ranges are statistically similar, then the engagement sequences in both Janus 
and JLink may be in agreement. 

The link between detection ranges and kill ranges focused the investigation into 
resolving the disparate kill range issue. As mentioned in the previous section, further 
analysis of the detection ranges, without firing artillery, has begun using the armored 
scenario in SWA. This ongoing investigation will also analyze the kill ranges to verify 
whether the kill ranges were indeed affected by the detection ranges. The assumption is 
that if the detection range issue is resolved, the kill ranges between Janus and JLink will 
become more similar. Presently, this hypothesis is being tested. 

The last result analyzed for kill ranges is the evidence of interaction between the 
environment and mode for mean red kill ranges. The interaction plot in Appendix E 
shows that the red force has the advantage in JLink of greater kill ranges, but the 
advantage is even more pronounced when fighting in HL terrain as compared to SWA 
terrain. The assumption that greater detection ranges in JLink lead to greater kill ranges 
in JLink accounts for part of the overall advantage. However, all ether factors being 
equal, one would expect the JLink kill range advantage to be consistent between 
scenarios since the JLink detection range advantage is consistent between scenarios. 
Because there is a larger red kill range difference in the HL as compared to SWA, further 
investigation is needed to identify the factor causing the difference. 
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One likely factor causing the difference in red kill ranges between environments 
and mode is the different terrain features of HL and SWA. Terrain features affect line of 
sight, which then affect detections, and subsequently the entire engagement process 
leading to rounds fired. Further testing should focus on this factor by simulating battalion 
size forces attacking under these two different environments. The issue of detection 
ranges should be resolved first, however, in order to isolate the factors causing 
differences in kill ranges. 

Overall, JLink’s current configuration returns kill ranges which are statistically 
similar to those in Janus in six of 12 cases. These results indicate that JLink provides a 
setting which generally replicates the kill ranges of different types of units fighting in 
different environments. In particular, JLink appears quite accurate in producing 
comparable kill ranges for armored coalition scenarios fighting on the SWA terrain, and 
attacking light infantry forces fighting on both HL and SWA terrain. These results also 
indicate that the JLink battlefield produces fairly accurate kill ranges for units conducting 
different types of missions, in this case an attacking red force against a defending blue 
force. Once further testing is complete, and the issue of detection range is resolved, one 
would expect JLink to produce kill ranges even closer to those of Janus. 

E. ROUNDS FIRED (BLUE/RED) 

The next MOP analyzed was rounds fired, analyzed separately by blue force and 
red force, using the data from Appendix B. The ANOVA returned a p-value of .96 for 
red rounds fired, which indicates that JLink is providing a battlefield setting which 
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replicates the average number of rounds fired comparable to Janus for a battalion-size 
armored or infantry force in the offense, under different environmental conditions. 

The results of the two-sample f-test and the Wilcoxon test for blue rounds fired 
were mutually supporting, although the r-test was not performed on several data sets 
because they did not satisfy t-test assumptions, primarily due to wide differences in 
variance between populations. The tests rejected the null hypothesis that the average blue 
rounds fired are the same between modes in four instances ; 

1. Blue Rounds Fired in Scenario #1, SWA 

2. Blue Rounds Fired in Scenario #1, HL 

3. Blue Rounds Fired in Scenario #2, SWA 

4. Blue Rounds Fired in Scenario #3, SWA 

The ANOVA supports these findings as indicated by the F-values for Mode in Appendix 
E. The F-value of 11.929 for the blue rounds fired is interpreted as, the average blue 
rounds fired among all Janus scenarios are statistically different than the average blue 
rounds fired among all JLink scenarios. 

The interaction between environment and mode for blue rounds fired indicates 
that the difference in blue rounds fired between Janus and JLink is even greater when 
placed in the SWA environment. As mentioned previously, investigation into the cause 
of this interaction must focus on the competing terrain of HL and SWA. Terrain affects 
line of sight and ultimately the engagement sequence and rounds fired. Once the 
detection range issue is resolved, further testing should focus on simulating units in the 
defense under different environments. 
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Next, the four scenarios which rejected the null hypothesis were analyzed to 
determine if the differences in the number of rounds fired between Janus and JLink 
satisfied the accepted tolerance. Recall that the accepted tolerance for rounds fired was 
A = 10% of the total Janus rounds fired. Applying the method illustrated in Figure 7 of 
comparing the MOP’s confidence interval to the determined A - value resulted in none of 
the four being within acceptable tolerance. 

Based on the results of the statistical tests for rounds fired, JLink provides an 
accurate replication of the battlefield that generates an average number of rounds fired 
comparable to Janus for a battalion-size unit attacking under different environmental 
conditions. JLink also provides a fare replication of the battlefield that generates similar 
average rounds fired for a company-size unit defending in HL terrain. 

Further, in reference to the discussion of the engagement process in paragraph D 
above, the number of rounds fired could become more similar between Janus and Jlink 
once the detection range and kill range issues are resolved. Since JLink consistently had 
greater detection ranges and kill ranges than Janus in this experiment, one would 
conclude that the number of rounds fired were affected by the disparity given that all 
three MOPs are linked in the engagement process. The ongoing experiment of the 
armored scenario in SWA, stripped of all artillery munitions, will be a valuable study to 
analyze the effect that accurate detection ranges have on rounds fired, especially since 
both the blue and red forces in the armored scenario in SWA appeared to have different 
averages of numbers of rounds fired. 
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F. 


LOSS EXCHANGE RATIO (LER) 


The LER is the ratio of red casualties to blue casualties, and offers a quantitative 
method to assess the overall outcome of a battle. Given that the factors affecting the ratio 
are accurately portrayed, LERs determined to be statistically similar provide strong 
supporting evidence that JLink is accurately portraying the entire scenario. In this case, 
both the r-test and Wilcoxon test failed to reject the null hypothesis in four of the six 
scenarios when testing the LERs. This indicates that the Janus and JLink LERs for these 
four scenarios are similar. In scenario #1, SWA and scenario #3, SWA, the hypothesis of 
equal LERs was rejected. Although, on four of six occasions, the null hypothesis was not 
rejected, an analyst must be cautious to accept the conclusion that JLink is accurately 
portraying the overall battle under these four conditions based on these results. 

The question posed then is, why are two of the six not statistically similar? The 
LER is a function of several other factors, primarily kills and detections. Previous 
analysis has shown that kills due to chemical artillery were not accurately portrayed by 
JLink during the experiment. Chemical kills is a factor of the LER, and inaccuracies with 
simulating those kills will adversely affect the LER for that scenario. Additionally, 
discussion of the detection ranges concluded that JLink did not satisfactorily replicate the 
detection ranges in this experiment, which is the another major contributing factor of the 
LER. The chemical kills and detection range issues alone could lead to dissimilar LERs 
between Janus and JLink. 

Given that at least two inputs to the LER equation may not have been accurate, it 
is reassuring that four of the six combinations were within statistical tolerance. There is a 
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strong likelihood that once the artillery delivery methods in JLink are identical to Janus, 
the ongoing investigations into chemical kills and detection ranges may be resolved. 
After these issues are resolved, this MOP should be analyzed again. One would expect 
the LERs to become more similar for all scenarios once these major factors in the LER 
equation are corrected. 
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VI. CONCLUSIONS AND RECOMMENDATIONS 


This chapter draws conclusions from Chapter V, assessing the current state of 
JLink in terms of varied scenarios, coalition warfare, and specific combat functions. The 
chapter also provides recommendations for further study of JLink issues presented in this 
thesis. 


A. CONCLUSIONS 

1. Varied Scenarios 

Chapter II describes varied scenarios as different types of combat units in the 
offense and defense, negotiating obstacles, under different environmental conditions. 

Such scenarios had yet to be tested in JLink to the extent tested in this thesis. To assess 
whether JLink replicates these scenarios in a way similar to Janus, the MOPs detection 
range, kill range, rounds fired, and LER were analyzed. The first three MOPs allowed for 
analysis of the basic procedures of an engagement, which is critical to arriving at valid 
conclusions about scenario fidelity. The LER offered a quantitative method to assess the 
overall scenario. 

Although JLink did not accurately replicate detection ranges in this experiment, 
the related MOPs of kill range and rounds fired fared well in the overall analysis. Also, 
statistical tests concluded that four of the six JLink LERs were statistically similar to the 
corresponding Janus LERs. These results support the conclusion that JLink is robust 
enough to sufficiently replicate varied scenarios in a manner similar to Janus, despite 
inaccurate inputs in detection ranges. 
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2 . 


Coalition Warfare 


The same MOPs used for drawing conclusions about varied scenarios 
were used to assess JLink’s ability to portray coalition warfare. Notwithstanding the 
detection range inaccuracies, all MOPs analyzed for scenario #2, armored coalition in 
both HL and SWA environments, failed to reject the null hypothesis, except for red kill 
ranges in HL and blue rounds fired in SWA. Given these results, the thesis concludes 
that JLink produces similar results to stand-alone Janus when simulating coalition 
warfare. 


3. Specific Combat Functions 

a. FASCAM 

The MOP used to assess JLink FASCAM fidelity was the number of kills 
resulting from FASCAM minefields. Although the statistical results for FASCAM 
rejected the null hypothesis only once, the trend towards consistently higher Janus 
FASCAM kills indicates that Janus FASCAM may be more lethal than JLink FASCAM. 
Despite the trend, however, JLink appears to provide a fair representation of FASCAM. 
A stronger conclusion must be borne out of additional FASCAM testing which isolates 
the minefield. 


b. Chemical Artillery 

The MOP used to assess JLink chemical artillery was the number of kills 
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generated by chemical artillery rounds. In all cases, the statistical tests rejected the null 
hypothesis that the average number of Janus chemical kills were the same as the average 
number of JLink chemical kills. Given these results, the thesis concludes that JLink’s 
configuration during the experiment did not accurately replicate the effects of Janus 
chemical artillery. 

The investigation into the cause of the disparate chemical data resulted in a 
solution which changed the delivery of JLink chemical artillery from an individual round 
method to a volley method. Changing to the volley method and rerunning the armored 
scenario in SWA produced very similar chemical kills between Janus and JLink. The 
conclusion is that JLink replicates the effects of Janus chemical artillery for an armored 
scenario in SWA terrain when fired in volley. Further, the volley method must be used 
whenever a user simulates a JLink to JLink battle. 

B. RECOMMENDATIONS 

1. Artillery Code Upgrade 

After investigating the cause of the detection range disparities, a brief study was 
conducted without artillery to support the hypothesis that different artillery delivery 
methods contributed to the different detection results. The study, which involved the 
armored scenario in SWA, proved conclusive. Without artillery, the detection ranges 
were statistically similar. The conclusion is that different artillery delivery methods 
produce different detection ranges. 

The recommendation is that JLink follow the volley method used in Janus 
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stand-alone, and that code be rewritten in JLink to fire artillery using the volley method 
when two JLink players are involved in a simulation. 

2. Further Testing of Detection Ranges, Kill Ranges, Rnds Fired, LERs 

Once the JLink code is rewritten using the volley method for artillery, further tests 
are necessary using various scenarios and environments to confirm that JLink scenarios 
are generating similar detection ranges as compared to Janus scenarios. 

Also, the kill ranges for these scenarios should be analyzed to ensure JLink 
scenarios are producing similar kill ranges and LERs as compared to Janus. The 
assumption is that detection ranges significantly affect kill ranges, based on the 
discussion of the engagement sequence. Once the detection issue is resolved, one would 
expect the kill ranges to be more aligned between Janus and JLink. Then, further testing 
should focus on the kill range results of a battalion-size unit attacking in the HL and 
SWA terrain in order to identify the source of interaction between environment and 
mode. 

Rounds fired should also be analyzed in the re-run scenarios. Again, given the 
engagement sequence, one would expect the average rounds fired to be more similar 
between Janus and JLink once detection ranges and kill ranges are corrected. Additional 
testing should focus on units in the defense under both environments to isolate the cause 
of environment and mode interaction for blue rounds fired. 

Since detection ranges, kill ranges, and rounds fired are inputs into the LER 
equation, one would expect the JLink LERs to be more in line with the Janus LERs once 
the detection ranges and kill ranges generate similar results between the two modes. 
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Testing and resolving these issues will lead to a more robust JLink system, capable of 
accurately portraying more diverse scenarios. 

3. Further Testing of FASCAM and Chemical Artillery 

The trend in the data consistently showing greater Janus FASCAM kills requires 
that FASCAM be investigated further. Also, chemical artillery must be tested using 
scenarios other than the previously tested armored scenario in SWA to confirm that the 
JLink code upgrades for chemical rounds sufficiently replicates chemical kills between 
the two modes. Improvements in both FASCAM and chemical artillery will diversify 
JLink and increase its value as a training and analytical tool. 

C. FINAL COMMENTS 

This thesis provides the latest assessment of JLink fidelity. The conclusions and 
recommendations of this thesis will improve distributed interoperability. However, in 
order to achieve the final goal of 100% interoperability between JLink systems, continued 
testing is necessary on the topics mentioned in this thesis and future topics generated by 
the evolving Jlink system. 
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APPENDIX A. RAW DATA 


SCENARIO #1, SWA RESULTS 



FASCAM Kills 

Chem Kills 

LER 

Detection Ranges 

Kill Ranges 

Rounds Fired 

Run# 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

1 

4 

4 

4 

0 

1.95 

1.08 

3.400 

3.844 

2.411 

2.636 

236 

248 

2 

2 

2 

6 

0 

2.11 

0.81 

3.351 

3.814 

2.493 

2.638 

119 

235 

3 

0 

1 

5 

0 

2.59 

0.96 

3.278 

3.862 

2.632 

2.673 

247 

201 

4 

6 

0 

4 

0 

2.14 

0.96 

3.294 

3.905 

2.749 

2.876 

183 

163 

5 

2 

3 

4 

0 

2.47 

1.09 

3.298 

3.826 

2.464 

2.671 

255 

191 

6 

1 

2 

4 

1 

1.95 

1.19 

3.394 

3.813 

2.668 

2.641 

150 

302 

7 

3 

2 

4 

0 

2 

1 

3.260 

3.885 

2.391 

2.634 

225 

228 

8 

3 

3 

4 

1 

2.61 

1.41 

3.232 

3.753 

2.739 

3.057 

205 

223 

9 

3 

1 

5 

1 

2.05 

1.39 

3.338 

3.765 

2.561 

2.746 

187 

246 

10 

3 

2 

4 

0 

1.95 

1.72 

3.425 

3.647 

2.650 

2.830 

147 

244 

XBAR 

2.7 

2 

4.4 

0.3 

2.182 

1.161 

3.327 

3.811 

2.576 

2.740 

195.4 

228.1 

VAR 

2.68 

1.33 

0.49 

0.23 

0.072 

0.074 

0.004 

0.006 

0.017 

0.020 

2144.04 

1428.1 

STDEV 

1.64 

1.15 

0.70 

0.48 

0.269 

0.273 

0.065 

0.075 

0.131 

0.141 

46.30 

37.79 


SCENARIO #1, HL RESULTS 



FASCAM Kills 

Chem Kills 

LER 

Detection Ranges 

Kill Ranges 

Rounds Fired 

Run# 

Janus 

JLINK 

Janus 

JLINK 

Janus 

JLINK 

Janus 

JLINK 

Janus 

JLINK 

Janus 

JLINK 

1 

4 

3 

8 

3 

2.09 

1.69 

3.027 

3.544 

2.406 

2.983 

546 

328 

2 

3 

1 

12 

4 

1.46 

1.52 

3.099 

3.772 

2.388 

2.711 

267 

256 

3 

6 

2 

13 

2 

1.36 

1.8 

3.089 

3.733 

2.428 

2.878 

454 

370 

4 

3 

1 

13 

4 

1.29 

1.43 

2.889 

3.581 

2.342 

2.946 

815 

551 

5 

1 

2 

14 

1 

1.95 

1.32 

2.945 

3.601 

2.311 

2.905 

570 

393 

6 

0 

1 

13 

4 

1.56 

1.87 

2.843 

3.455 

2.179 

2.687 

418 

354 

7 

2 

0 

11 

2 

1.37 

1.19 

2.913 

3.680 

2.347 

2.665 

705 

410 

8 

2 

1 

13 

2 

1.6 

1.52 

2.914 

3.305 

2.085 

2.832 

669 

453 

9 

4 

2 

14 

2 

2.5 

1.32 

2.853 

3.699 

2.532 

2.663 

839 

320 

10 

4 

2 

12 

1 

1.41 

1.39 

2.743 

3.552 

2.346 

2.858 

1159 

384 

XBAR 

2.9 

1.5 

12.3 

2.5 

1.659 

1.505 

2.931 

3.592 

2.337 

2.813 

644.2 

381.9 

VAR 

2.99 

0.72 

3.12 

1.39 

0.156 

0.049 

0.013 

0.020 

0.016 

0.015 

64580.18 

6452.77 

STDEV 

1.73 

0.85 

1.77 

1.18 

0.395 

0.221 

0.112 

0.140 

0.126 

0.121 

254.13 

80.33 


SCENARIO #2, SWA RESULTS 



FASCAM Kills 

Chem Kills 

LER 

Detection Ranges 

Kill Ranges 

Rounds Fired 

Run# 

Janus 

JLINK 

Janus 

JLINK 

Janus 

JLINK 

Janus 

JLINK 

Janus 

JLINK 

Janus 

JLINK 

1 

2 

3 

4 

0 

1.26 

1.4 

3.196 

3.364 

2.574 

2.635 

141 

147 

2 

1 

0 

3 

0 

1.13 

0.75 

3.230 

3.415 

2.360 

2.640 

136 

245 

3 

4 

1 

5 

0 

1.52 

1.17 

3.069 

3.511 

2.746 

2.528 

215 

189 

4 

4 

2 

4 

1 

1.67 

1.21 

2.955 

3.263 

2.447 

2.563 

140 

231 

5 

2 

3 

3 

0 

1.38 

1.65 

3.018 

3.379 

2.761 

2.735 

151 

227 

6 

4 

1 

3 

0 

1.48 

1.05 

3.149 

3.234 

2.499 

2.521 

387 

152 

7 

4 

0 

4 

0 

1.48 

1.22 

3.086 

3.297 

2.462 

2.527 

182 

228 

8 

2 

3 

4 

0 

1.29 

1.33 

3.157 

3.395 

2.358 

2.484 

150 

184 

9 

0 

2 

4 

0 

1.26 

1.15 

3.217 

3.366 

2.553 

2.960 

304 

180 

10 

0 

0 

4 

0 

1.3 

0.81 

3.135 

3.200 

2.469 

3.262 

199 

227 

XBAR 

2.3 

1.5 

3.8 

0.1 

1.377 

1.174 

3.121 

3.342 

2.523 

2.685 

200.5 

201 

VAR 

2.68 

1.61 

0.4 

0.1 

0.025 

0.070 

0.008 

0.009 

0.020 

0.061 

6934.5 

1234.22 

STDEV 

1.64 

1.27 

0.63 

0.32 

0.160 

0.265 

0.089 

0.094 

0.140 

0.247 

83.27 

35.13 
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SCENARIO #2, HL RESULTS 



FASCAM Kills 

Chem Kills 

LER 

Detection Ranges 

Kill Ranges 

Rounds Fired 

Run# 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JLINK 

Janus 

JLINK 

Janus 

JLINK 

Janus 

JLINK 

1 

1 

0 

11 

2 

1.3 

2.4 

2.106 

2.398 

2.247 

2.745 

385 

353 

2 

4 

1 

12 

3 

1.07 

1.64 

2.054 

2.594 

2.414 

2.715 

516 

362 

3 

2 

1 

9 

2 

1.48 

2.1 

2.115 

2.370 

2.294 

2.677 

607 

452 

4 

6 

0 

14 

2 

1.58 

2 

2.105 

2.515 

2.281 

2.639 

448 

312 

5 

2 

0 

14 

5 

1.78 

1.67 

2.150 

2.519 

2.463 

2.825 

478 

520 

6 

2 

1 

14 

2 

1.46 

1.92 

2.085 

2.496 

2.182 

2.688 

591 

711 

7 

2 

1 

12 

3 

1.71 

1.5 

2.084 

2.322 

2.167 

2.881 

795 

344 

8 

2 

0 

13 

4 

1.5 

1.18 

2.081 

2.400 

2.115 

2.778 

813 

688 

9 

4 

1 

13 

3 

1.44 

2.1 

2.140 

2.401 

2.164 

2.609 

711 

522 

10 

1 

1 

11 

3 

1.57 

1.84 

2.105 

2.572 

2.489 

2.522 

604 

591 

XBAR 

2.6 

0.6 

12.3 

2.9 

1.489 

1.835 

2.103 

2.459 

2.282 

2.708 

594.8 

485.5 

VAR 

2.49 

0.27 

2.68 

0.99 

0.040 

0.122 

0.001 

0.008 

0.018 

0.011 

20762.18 

21062.72 

STDEV 

1.58 

0.52 

1.64 

0.99 

0.201 

0.350 

0.028 

0.092 

0.133 

0.106 

144.09 

145.13 


SCENARIO #3, SWA RESULTS 



FASCAM Kills 

Chem Kills 

LER 

Detection Ranges 

Kill Ranges 

Rounds Fired 

Run # 

Janus 

JLINK 

Janus 

JLINK 

Janus 

JLINK 

Janus 

JLINK 

Janus 

JUNK 

Janus 

JLINK 

1 

2 

0 

9 

0 

1.28 

0.94 

1.935 

2.036 

1.334 

1.526 

2640 

2396 

2 

0 

2 

15 

0 

1.36 

0.88 

1.932 

2.028 

1.437 

1.528 

2486 

2253 

3 

0 

0 

12 

0 

1.53 

0.79 

1.956 

2.050 

1.378 

1.493 

2417 

2202 

4 

1 

0 

6 

0 

2.28 

0.95 

1.895 

2.060 

1.309 

1.531 

2890 

2309 

5 

0 

0 

9 

3 

1.92 

1.06 

1.937 

2.011 

1.312 

1.449 

2712 

2163 

6 

0 

0 

12 

3 

1.39 

0.77 

1.964 

2.055 

1.361 

1.560 

2553 

2111 

7 

5 

0 

12 

0 

2.47 

0.95 

1.906 

2.026 

1.408 

1.468 

2981 

2292 

8 

0 

0 

6 

0 

1.17 

0.92 

1.931 

2.050 

1.436 

1.492 

2339 

2156 

9 

2 

0 

6 

0 

0.92 

1.03 

1.963 

2.057 

1.441 

1.484 

2288 

2207 

10 

1 

0 

12 

0 

1.25 

1.08 

1.988 

2.032 

1.371 

1.485 

2650 

2314 

XBAR 

1.1 

0.2 

9.9 

0.6 

1.557 

0.937 

1.941 

2.041 

1.379 

1.501 

2595.6 

2240.3 

VAR 

2.54 

0.4 

10.1 

1.6 

0.254 

0.011 

0.001 

0.000 

0.003 

0.001 

51096.71 

7733.79 

STDEV 

1.60 

0.63 

3.18 

1.26 

0.504 

0.104 

0.028 

0.016 

0.051 

0.034 

226.05 

87.94 


SCENARIO #3* HL RESULTS 



FASCAM Kills 

Chem Kills 

LER 

Detection Ranges 

Kill Ranges 

Rounds Fired 

Run # 

Janus 

JLINK 

Janus 

JLINK 

Janus 

JLINK 

Janus 

JLINK 

Janus 

JLINK 

Janus 

JLINK 

1 

2 

1 

69 

33 

5.63 

2.22 

1.588 

1.713 

1.323 

1.443 

2815 

2896 

2 

0 

1 

69 

33 

2.71 

2.48 

1.610 

1.686 

1.294 

1.396 

2383 

2713 

3 

0 

1 

69 

48 

2.72 

2.49 

1.630 

1.677 

1.303 

1.374 

2895 

3310 

4 

1 

0 

69 

42 

3.74 

3.65 

1.649 

1.658 

1.328 

1.306 

3308 

2921 

5 

2 

1 

72 

33 

3.51 

2.06 

1.579 

1.716 

1.228 

1.495 

2412 

3121 

6 

0 

1 

69 

21 

2.18 

2.6 

1.665 

1.667 

1.372 

1.369 

2786 

2823 

7 

0 

0 

72 

36 

2.59 

1.32 

1.590 

1.770 

1.382 

1.511 

2284 

2978 

8 

1 

1 

69 

30 

2.42 

2.35 

1.605 

1.725 

1.250 

1.416 

2816 

3246 

9 

3 

0 

69 

39 

2.93 

2.9 

1.589 

1.664 

1.376 

1.345 

2727 

2565 

10 

1 

0 

69 

51 

2.69 

1.53 

1.593 

1.733 

1.271 

1.567 

2705 

2782 

XBAR 

1 

0.6 

69.6 

36.6 

3.112 

2.36 

1.610 

1.701 

1.313 

1.422 

2713.1 

2935.5 

VAR 

1.11 

0.27 

1.6 

77.6 

1.004 

0.436 

0.001 

0.001 

0.003 

0.007 

88588.1 

55293.61 

STDEV 

1.05 

0.52 

1.26 

8.81 

1.002 

0.660 

0.029 

0.037 

0.054 

0.082 

297.64 

235.15 
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APPENDIX B. RAW DATA SEPARATED BY SIDE 


SCENARIO #1, SWA RESULTS 



Det Ranges Blue 

Det Ranges Red 

Kill Ranges Blue 

Kill Ranges Red 

Rnds Fired Blue 

Rnds Fired Red 

Run# 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

1 

3.847 

4.231 

2.381 

3.398 

1.986 

2.356 

2.704 

2.987 

60 

27 

176 

221 

2 

3.707 

4.139 

2.344 

3.393 

1.903 

2.413 

2.805 

3.028 

54 

23 

65 

212 

3 

3.616 

4.220 

2.412 

3.510 

1.952 

2.343 

2.901 

3.064 

172 

29 

75 

172 

4 

3.656 

4.274 

2.480 

3.425 

2.020 

2.393 

3.153 

3.404 

87 

29 

96 

134 

5 

3.570 

4.164 

2.661 

3.452 

1.862 

2.559 

2.725 

2.826 

102 

23 

153 

168 

6 

3.825 

4.068 

2.308 

3.538 

1.905 

2.353 

3.083 

2.930 

63 

55 

87 

247 

7 

3.570 

4.245 

2.593 

3.502 

1.929 

2.381 

2.661 

2.909 

108 

30 

117 

198 

8 

3.651 

4.187 

2.308 

3.108 

3.049 

2.458 

2.613 

3.630 

85 

30 

120 

193 

9 

3.764 

4.087 

2.414 

3.418 

2.039 

2.404 

2.860 

3.037 

64 

38 

123 

208 

10 

3.684 

4.094 

2.800 

3.148 

2.096 

2.276 

2.968 

3.185 

58 

44 

89 

200 

XBAR 

3.689 

4.171 

2.470 

3.389 

2.074 

2.394 

2.847 

3.100 

85.3 

32.8 

110.1 

195.3 

VAR 

0.010 

0.005 

0.027 

0.021 

0.123 

0.006 

0.033 

0.061 

1292.23 

101.73 

1215.43 

983.79 

STDEV 

0.098 

0.072 

0.165 

0.146 

0.350 

0.076 

0.181 

0.246 

35.95 

10.09 

34.86 

31.37 







SCENARIO #1, HL RESULTS 







Det Ranges Blue 

Det Ranges Red 

Kill Ranges Blue 

Kill Ranges Red 

Rnds Fired Blue 

Rnds Fired Red 

Run # 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JLINK 

Janus 

JLINK 

Janus 

JLINK 

Janus 

JLINK 

1 

3.389 

3.845 

2.022 

2.959 

2.000 

2.790 

2.629 

3.112 

171 

65 

375 

263 

2 

3.419 

4.047 

2.076 

3.078 

1.821 

2.302 

2.835 

3.003 

87 

59 

180 

197 

3 

3.419 

4.060 

2.068 

3.092 

2.249 

2.580 

2.590 

3.060 

150 

65 

304 

305 

4 

3.259 

3.970 

2.142 

2.886 

2.138 

3.019 

2.534 

2.889 

78 

59 

737 

492 

5 

3.362 

3.901 

1.880 

3.086 

1.920 

2.689 

2.534 

3.078 

149 

60 

421 

333 

6 

3.211 

3.913 

2.070 

2.530 

1.726 

2.293 

2.470 

2.908 

147 

62 

271 

292 

7 

3.258 

4.060 

2.103 

2.797 

1.851 

2.427 

2.740 

2.866 

86 

49 

619 

361 

8 

3.349 

3.765 

1.892 

2.686 

1.617 

2.601 

2.410 

3.000 

87 

62 

582 

391 

9 

3.338 

4.071 

1.849 

2.835 

2.248 

2.237 

2.664 

3.014 

90 

53 

749 

267 

10 

3.296 

3.873 

1.903 

2.871 

1.978 

2.623 

2.670 

3.045 

78 

47 

1081 

337 

XBAR 

3.330 

3.951 

2.000 

2.882 

1.955 

2.556 

2.607 

2.997 

112.3 

58.1 

531.9 

323.8 

VAR 

0.005 

0.012 

0.012 

0.034 

0.045 

0.061 

0.016 

0.007 

1360.01 

40.32 

75729.21 

6535.07 

STDEV 

0.072 

0.107 

0.108 

0.183 

0.212 

0.246 

0.127 

0.084 

36.88 

6.35 

275.19 

80.84 






SCENARIO #2, SWA 

RESULTS 








Det Ranges Blue 

Det Ranges Red 

Kill Ranges Blue 

Kill Ranges Red 

Rnds Fired Blue 

Rnds Fired Red 

Run# 

Janus 

JLINK 

Janus 

JLINK 

Janus 

JLINK 

Janus 

JLINK 

Janus 

JLINK 

Janus 

JLINK 

1 

3.841 

4.128 

2.316 

3.271 

1.871 

2.098 

3.277 

3.115 

59 

35 

82 

112 

2 

3.718 

4.118 

2.218 

3.365 

1.788 

2.205 

3.019 

3.026 

34 

74 

102 

171 

3 

3.685 

3.992 

2.292 

3.325 

2.687 

2.098 

2.796 

2.958 

42 

29 

173 

160 

4 

3.614 

4.075 

2.219 

3.166 

1.742 

2.197 

2.993 

2.902 

49 

32 

91 

199 

5 

3.584 

4.151 

2.115 

3.376 

2.672 

2.179 

2.840 

3.162 

72 

38 

79 

189 

6 

3.173 

4.090 

2.030 

2.998 

1.766 

2.246 

3.173 

2.838 

52 

28 

335 

124 

7 

3.784 

4.103 

2.328 

3.219 

1.864 

2.121 

2.929 

2.901 

74 

36 

108 

192 

8 

3.611 

4.090 

2.091 

3.234 

1.771 

2.110 

2.900 

2.843 

62 

30 

88 

154 

9 

3.558 

4.176 

2.208 

3.492 

1.790 

2.375 

3.179 

3.609 

70 

22 

234 

158 

10 

3.625 

3.812 

2.302 

2.977 

1.916 

3.626 

2.940 

2.684 

51 

29 

148 

198 

XBAR 

3.619 

4.073 

2.212 

3.242 

1.987 

2.325 

3.005 

3.004 

56.5 

35.3 

144 

165.7 

VAR 

0.033 

0.011 

0.011 

0.027 

0.136 

0.216 

0.025 

0.065 

176.5 

206.01 

6945.78 

918.46 

STDEV 

0.181 

0.104 

0.103 

0.163 

0.369 

0.465 

0.158 

0.255 

13.29 

14.35 

83.34 

30.31 
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SCENARIO #2, HL RESULTS 



Del Ranges Blue 

Del Ranges Red 

Kill Ranges Blue 

Kill Ranges Red 

Rnds Fired Blue 

Rnds Fired Red 

Run # 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

1 

3.312 

3.900 

1.944 

2.913 

2.033 

1.738 

2.423 

3.164 

78 

91 

307 

262 

2 

3.573 

4.036 

1.907 

3.021 

2.073 

2.226 

2.797 

3.028 

41 

60 

475 

302 

3 

3.361 

3.991 

1.792 

2.577 

2.011 

1.741 

2.500 

3.145 

64 

59 

543 

393 

4 

3.301 

3.982 

1.866 

2.648 

1.683 

2.066 

2.737 

2.932 

83 

67 

367 

245 

5 

3.465 

3.904 

1.899 

2.709 

1.984 

2.599 

2.769 

2.963 

56 

81 

422 

439 

6 

3.245 

3.890 

2.025 

2.496 

1.718 

2.275 

2.541 

2.908 

67 

123 

524 

588 

7 

3.401 

4.074 

1.807 

2.474 

1.536 

2.544 

2.599 

3.112 

66 

56 

729 

288 

8 

3.338 

3.970 

1.839 

2.717 

1.612 

2.561 

2.492 

2.967 

116 

58 

697 

630 

9 

3.335 

3.838 

1.993 

2.227 

1.582 

1.887 

2.633 

2.970 

63 

72 

648 

450 

10 

3.286 

3.895 

1.806 

2.463 

2.083 

1.809 

2.765 

2.927 

82 

69 

522 

522 

XBAR 

3.362 

3.948 

1.888 

2.624 

1.831 

2.145 

2.626 

3.012 

71.6 

73.6 

523.4 

411.9 

VAR 

0.009 

0.006 

0.007 

0.054 

0.050 

0.120 

0.018 

0.009 

401.6 

424.04 

19137.16 

18999.88 

STDEV 

0.096 

0.074 

0.081 

0.232 

0.224 

0.346 

0.135 

0.095 

20.04 

20.59 

138.34 

137.84 


SCENARIO #3, SWA RESULTS 



Det Ranges Blue 

Det Ranges Red 

Kill Ranges Blue 

Kill Ranges Red 

Rnds Fired Blue 

Rnds Fired Red 

Run # 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

1 

2.050 

2.038 

1.811 

2.034 

1.359 

1.616 

1.313 

1.427 

875 

646 

1765 

1750 

2 

2.071 

2.070 

1.793 

1.990 

1.481 

1.611 

1.405 

1.433 

866 

628 

1620 

1625 

3 

2.057 

2.116 

1.821 

1.992 

1.596 

1.594 

1.235 

1.365 

972 

557 

1445 

1645 

4 

2.014 

2.068 

1.747 

2.052 

1.372 

1.683 

1.282 

1.370 

1295 

654 

1595 

1655 

5 

2.046 

2.034 

1.778 

1.989 

1.450 

1.635 

1.240 

1.271 

1137 

638 

1575 

1525 

6 

2.087 

2.107 

1.833 

2.007 

1.525 

1.609 

1.244 

1.495 

833 

546 

1720 

1565 

7 

2.027 

2.079 

1.746 

1.973 

1.510 

1.610 

1.366 

1.315 

1456 

617 

1525 

1675 

8 

2.058 

2.075 

1.785 

2.028 

1.540 

1.631 

1.345 

1.338 

749 

601 

1590 

1555 

9 

2.126 

2.066 

1.779 

2.048 

1.462 

1.659 

1.416 

1.313 

658 

637 

1630 

1570 

10 

2.138 

2.097 

1.817 

1.971 

1.476 

1.611 

1.286 

1.365 

855 

689 

1795 

1625 

XBAR 

2.067 

2.075 

1.791 

2.008 

1.477 

1.626 

1.313 

1.369 

969.6 

621.3 

1626 

1619 

VAR 

0.002 

0.001 

0.001 

0.001 

0.005 

0.001 

0.005 

0.004 

63081.38 

1896.46 

11621.11 

4510 

STDEV 

0.040 

0.027 

0.030 

0.030 

0.072 

0.027 

0.067 

0.067 

251.16 

43.55 

107.80 

67.16 


SCENARIO #3, HL RESULTS 



Det Ranges Blue 

Det Ranges Red 

Kill Ranges Blue 

Kill Ranges Red 

Rnds Fired Blue 

Rnds Fired Red 

Run # 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

1 

1.613 

1.718 

1.547 

1.702 

1.327 

1.377 

1.322 

1.474 

1960 

1276 

855 

1620 

2 

1.641 

1.707 

1.572 

1.652 

1.312 

1.359 

1.287 

1.412 

1068 

1303 

1315 

1410 

3 

1.665 

1.703 

1.586 

1.634 

1.312 

1.363 

1.300 

1.378 

1345 

1535 

1550 

1775 

4 

1.675 

1.676 

1.612 

1.626 

1.193 

1.420 

1.364 

1.275 

1833 

1636 

1475 

1285 

5 

1.614 

1.733 

1.536 

1.690 

1.319 

1.336 

1.202 

1.572 

1257 

1361 

1155 

1760 

6 

1.723 

1.677 

1.579 

1.651 

1.323 

1.376 

1.394 

1.366 

1151 

1268 

1635 

1555 

7 

1.632 

1.786 

1.538 

1.744 

1.399 

1.373 

1.376 

1.617 

1134 

1018 

1150 

1960 

8 

1.644 

1.747 

1.547 

1.688 

1.241 

1.362 

1.253 

1.438 

1131 

1486 

1685 

1760 

9 

1.623 

1.690 

1.543 

1.620 

1.299 

1.398 

1.403 

1.327 

1352 

1330 

1375 

1235 

10 

1.646 

1.753 

1.530 

1.709 

1.286 

1.386 

1.265 

1.687 

1245 

1202 

1460 

1580 

XBAR 

1.648 

1.719 

1.559 

1.672 

1.301 

1.375 

1.317 

1.455 

1347.6 

1341.5 

1365.5 

1594 

VAR 

0.001 

0.001 

0.001 

0.002 

0.003 

0.001 

0.004 

0.018 

93197.38 

31205.83 

64852.5 

53604.44 

STDEV 

0.033 

0.036 

0.027 

0.041 

0.055 

0.023 

0.067 

0.133 

305.28 

176.65 

254.66 

231.53 
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APPENDIX C. RAW DATA FOR SAME SEED JLINK RUNS 


SCENARIO #1, SWA JLINK RESULTS 


Seed# 

Run# 

FASCAM Kills 

Chem Kills 

LER 

Detection Ranges 

Kill Ranges 

Rounds Fired 

55 

1 

4 

0 

1.08 

3.844 

2.636 

248 

55 

2 

1 

1 

1 

3.816 

2.707 

260 

55 

3 

3 

0 

1.14 

3.851 

2.774 

158 

55 

4 

1 

0 

2.09 

3.705 

2.804 

252 

55 

5 

2 

0 

1.13 

3.853 

2.851 

159 

55 

6 

1 

0 

1.3 

3.786 

2.635 

182 

55 

7 

0 

0 

1.17 

3.809 

2.711 

257 

55 

8 

2 

0 

0.88 

3.822 

2.645 

196 

55 

9 

1 

0 

0.92 

3.837 

2.604 

224 

55 

10 

4 

0 

1.17 

3.956 

2.700 

230 


XBAR 

1.9 

0.1 

1.188 

3.828 

2.707 

216.6 


VAR 

1.88 

0.1 

0.116 

0.004 

0.007 

1595.82 


STDEV 

1.37 

0.32 

0.341 

0.063 

0.081 

39.95 


SCENARIO #1, HL JLINK RESULTS 


Seed# 

Run # 

FASCAM Kills 

Chem Kills 

LER 

Detection Ranges 

Kill Ranges 

Rounds Fired 

55 

1 

3 

3 

1.69 

3.544 

2.983 

328 

55 

2 

1 

1 

1.21 

3.531 

2.757 

377 

55 

3 

2 

3 

1.32 

3.583 

2.776 

248 

55 

4 

2 

2 

1.56 

3.624 

2.983 

219 

55 

5 

0 

2 

1.43 

3.787 

3.120 

306 

55 

6 

2 

2 

1.1 

3.633 

2.763 

466 

55 

7 

1 

4 

1.46 

3.438 

2.805 

307 

55 

8 

2 

4 

1.91 

3.583 

2.811 

302 

55 

9 

1 

1 

1.15 

3.689 

2.617 

212 

55 

10 

2 

3 

1.92 

3.691 

2.811 

277 


XBAR 

1.6 

2.5 

1.475 

3.610 

2.843 

304.2 


VAR 

0.71 

1.17 

0.087 

0.010 

0.021 

5746.62 


STDEV 

0.84 

1.08 

0.295 

0.098 

0.145 

75.81 


SCENARIO #2, SWA JLINK RESULTS 


Seed# 

Run # 

FASCAM Kills 

Chem Kills 

LER 

Detection Ranges 

Kill Ranges 

Rounds Fired 

55 

1 

3 

0 

1.4 

3.364 

2.635 

147 

55 

2 

0 

0 

1.17 

3.369 

2.859 

106 

55 

3 

0 

0 

1.19 

3.360 

2.441 

176 

55 

4 

2 

1 

1.2 

3.514 

2.587 

158 

55 

5 

2 

0 

1.35 

3.428 

2.550 

204 

55 

6 

2 

0 

1.26 

3.451 

3.480 

121 

55 

7 

0 

0 

1.32 

3.223 

2.500 

258 

55 

8 

1 

1 

1.17 

3.283 

2.831 

243 

55 

9 

1 

0 

0.96 

3.324 

2.539 

256 

55 

10 

0 

1 

1.31 

3.339 

2.467 

319 


XBAR 

1.1 

0.3 

1.233 

3.365 

2.689 

198.8 


VAR 

1.21 

0.23 

0.016 

0.007 

0.097 

4748.62 


STDEV 

1.10 

0.48 

0.125 

0.084 

0.312 

68.91 
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SCENARIO #2, HL JUNK RESULTS 


Seed# 

Run# 

FASCAM Kills 

Chem Kills 

LER 

Detection Ranges 

Kill Ranges 

Rounds Fired 

55 

1 

0 

2 

2.4 

2.398 

2.745 

353 

55 

2 

2 

1 

1.36 

2.254 

2.501 

527 

55 

3 

1 

1 

1.5 

2.189 

2.619 

362 

55 

4 

2 

2 

1.63 

2.469 

2.692 

319 

55 

5 

2 

0 

1.31 

2.264 

2.687 

426 

55 

6 

2 

3 

1.77 

2.435 

2.792 

450 

55 

7 

2 

4 

2.63 

2.486 

2.787 

461 

55 

8 

0 

1 

1.54 

2.518 

2.712 

336 

55 

9 

2 

2 

1.59 

2.284 

2.708 

531 

55 

10 

3 

2 

1.79 

2.415 

2.498 

549 


XBAR 

1.6 

1.8 

1.752 

2.371 

2.674 

431.4 


VAR 

0.93 

1.29 

0.188 

0.013 

0.011 

7426.49 


STDEV 

0.97 

1.14 

0.433 

0.114 

0.105 

86.18 


SCENARIO #3, SWA JUNK RESULTS 


Seed # 

Run # 

FASCAM Kills 

Chem Kills 

LER 

Detection Ranges 

Kill Ranges 

Rounds Fired 

55 

1 

0 

0 

0.94 

2.036 

1.526 

2396 

55 

2 

0 

0 

1.18 

2.030 

1.423 

2268 

55 

3 

0 

0 

0.87 

2.059 

1.508 

2126 

55 

4 

0 

0 

0.78 

2.053 

1.511 

2228 

55 

5 

0 

3 

1.03 

2.057 

1.413 

2212 

55 

6 

1 

0 

1.09 

2.060 

1.441 

2285 

55 

7 

0 

0 

0.95 

2.055 

1.482 

2340 

55 

8 

0 

0 

0.95 

2.088 

1.564 

2321 

55 

9 

0 

0 

0.95 

2.022 

1.520 

2215 

55 

10 

0 

0 

1 

2.046 

1.488 

2215 


XBAR 

0.1 

0.3 

0.974 

2.051 

1.488 

2260.6 


VAR 

0.1 

0.9 

0.012 

0.000 

0.002 

6070.71 


STDEV 

0.32 

0.95 

0.111 

0.019 

0.049 

77.91 


SCENARIO #3, HL JLINK RESULTS 


Seed # 

Run# 

FASCAM Kills 

Chem Kills 

LER 

Detection Ranges 

Kill Ranges 

Rounds Fired 

55 

1 

1 

33 

2.22 

1.713 

1.443 

2896 

55 

2 

1 

36 

2.29 

1.726 

1.481 

3137 

55 

3 

2 

42 

2.8 

1.735 

1.414 

3455 

55 

4 

0 

48 

2.81 

1.724 

1.420 

3066 

55 

5 

0 

33 

1.52 

1.738 

1.548 

2592 

55 

6 

0 

24 

1.58 

1.799 

1.493 

2835 

55 

7 

0 

36 

1.93 

1.696 

1.377 

2872 

55 

8 

2 

30 

2.19 

1.686 

1.322 

3085 

55 

9 

0 

33 

1.95 

1.692 

1.402 

2839 

55 

10 

0 

21 

1.87 

1.735 

1.406 

2798 


XBAR 

0.6 

33.6 

2.116 

1.724 

1.431 

2957.5 


VAR 

0.71 

61.6 

0.196 

0.001 

0.004 

56325.17 


STDEV 

0.84 

7.85 

0.442 

0.032 

0.064 

237.33 
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APPENDEXD. POWER DATA 


Scenario #1, SWA 


A 


<1) 

Power 

FASCAM 

1 

0.070 

22.54 

1.00 

Chemical Kills 

2 

0.601 

5.26 

1.00 

LER 

.3 

0.271 

1.75 

.72 

Detection Range 

.3 

0.070 

6.76 

1.00 

Kill Range 

.2 

0.136 

2.32 

.88 

Rounds Fired 

40 

42.262 

1.50 

.54 


Scenario#!, HL 


A 



Power 

FASCAM 

1 

1.362 

1.16 

.31 

Chemical Kills 

2 

1.502 

2.11 

.81 

LER 

.3 

0.320 

1.48 

.54 

Detection Range 

.3 

0.127 

3.73 

1.00 

Kill Range 

.2 

0.124 

2.56 

.93 

Rounds Fired 

130 

188.458 

1.09 

.29 


Scenario #2, SWA 


A 


<!> 

Power 

FASCAM 

1 

1.464 

1.08 

.28 

Chemical Kills 

2 

0.5 

6.33 

1.00 

LER 

.3 

0.219 

2.17 

.83 

Detection Range 

.3 

0.091 

5.19 

1.00 

Kill Range 

.2 

0.200 

1.58 

.56 

Rounds Fired 

40 

63.909 

0.99 

.25 
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Scenario #2, HL 


A 


<t) 

Power 

FASCAM 

1 

1.174 

1.35 

.40 

Chemical Kills 

2 

1.354 

2.34 

.88 

LER 

.3 

0.285 

1.66 

.62 

Detection Range 

.3 

0.0681 

6.96 

1.00 

Kill Range 

.2 

0.120 

2.63 

.94 

Rounds Fired 

120 

144.611 

1.31 

.40 


Scenario #3, SWA 


A 


<t) 

Power 

FASCAM 

1 

1.213 

1.30 

.40 

Chemical Kills 

20 

2.419 

13.07 

1.00 

LER 

.3 

0.364 

1.30 

.40 

Detection Range 

.2 

0.0228 

13.90 

1.00 

Kill Range 

.05 

0.043 

1.84 

.69 

Rounds Fired 

520 

171.509 

4.79 

1.00 


Scenario #3, HL 


A 


<|) 

Power 

FASCAM 

1 

0.830 

1.91 

.72 

Chemical Kills 

20 

6.293 

5.03 

1.00 

LER 

.3 

0.849 

0.56 

<.20 

Detection Range 

.2 

0.033 

9.60 

1.00 

Kill Range 

.05 

0.069 

1.14 

.31 

Rounds Fired 

540 

268.218 

3.18 

.986 
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APPENDIX E. ANALYSIS OF VARIANCE (ANOVA) RESULTS 


ANOVA FOR FASCAM KILLS 

Analysis of Variance Table 
Response: MOE 


Terms added sequentially (first to last) 



Df 

Sum of Sq 

Mean Sq 

F Value 

Seen 

2 

49.7167 

24.85833 

15.62689 

Env 

1 

0.3000 

0.30000 

0.18859 

Mode 

1 

32.0333 

32.03333 

20.13737 

Seen:Env 

2 

1.0500 

0.52500 

0.33003 

Seen:Mode 

2 

2.8167 

1.40833 

0.88533 

Env:Mode 

1 

1.6333 

1.63333 

1.02678 

Seen:Env:Mode 

2 

3.8167 

1.90833 

1.19965 

Residuals 

108 

171.8000 

1.59074 



Pr (F) 
0.0000011 
0.6649580 
0.0000181 
0.7196211 
0.4155499 
0.3131820 
0.3052820 


Interaction plots for scenariorenvironment, scenario:mode, and environmentrmode. 



93 
















ANOVA FOR CHEMICAL KILLS 


Analysis of Variance Table 
Response: MOE 


Terms added sequentially (first to last) 


Seen 

Env 

Mode 

Seen:Env 

Seen:Mode 

Env:Mode 

Seen:Env:Mode 

Residuals 


Df 

2 

1 

1 

2 

2 

1 

2 

108 


Sum of Sq 

15811.47 
11427.01 

4002.07 

12043.47 
1383 .20 
1026.68 

540.00 

902.70 


Mean Sq 

7905.73 
11427.01 

4002.07 

6021.73 
691.60 

1026.68 
270.00 
8.36 


F Value 
945.850 
1367.140 
478.813 
720.447 
82.744 
122.833 
32.303 


Pr (F) 
O.OOOOOOe+000 
O.OOOOOOe+000 
O.OOOOOOe+000 
0.OOOOOOe+000 
O.OOOOOOe+000 
0.OOOOOOe+000 
1.008882e-011 


Interaction plots for scenariorenvironment, scenariormode, and environment:mode. 
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ANOVA FOR LERs 


Analysis of Variance Table 
Response: MOE 


Terms added sequentially (first to last) 



Df 

Sum of Sq 

Mean Sq 

F Value 

Seen 

2 

5.75032 

2.87516 

14.90381 

Env 

1 

10.63265 

10.63265 

55.11590 

Mode 

1 

4.81601 

4.81601 

24.96450 

Seen:Env 

2 

13.11248 

6.55624 

33.98523 

Seen:Mode 

2 

3.39263 

1.69632 

8.79310 

Env:Mode 

1 

1.37388 

1.37388 

7.12171 

Seen:Env:Mode 

2 

1.30240 

0.65120 

3.37560 

Residuals 

108 

20.83476 

0.19291 



Pr (F) 

0.00000192 
0.00000000 
0.00000226 
0.00000000 
0.00028975 
0.00878978 
0.03784347 


Interaction plots for scenario environment, scenario:mode, and environment:mode. 
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ANOVA for Detection Range Blue 


Analysis of Variance Table 
Response; MOE 


Terms added sequentially (first to last) 


Seen 

Env 

Mode 

Seen:Env 

Seen:Mode 

Env:Mode 

Seen:Env:Mode 

Residuals 


Df Sum of Sq 
2 95.34164 

1 2.51797 

1 4.11438 

2 0.19275 

2 1.64627 

1 0.09369 

2 0.00861 

108 0.85094 


Mean Sq F V 
47.67082 6050 
2.51797 319 

4.11438 522 

0.09638 12 

0.82313 104 

0.09369 11 

0.00431 0 

0.00788 


lue Pr(F) 
279 0.0000000 
576 0.0000000 
188 0.0000000 
232 0.0000163 
470 0.0000000 
892 0.0008050 
547 0.5805110 


Interaction plots for scenario:environment, scenario:mode, and environment:mode. 



Scenario * Mode Interaction 


2 3 

Scenario « 


Environment • Mode Interaction 



Jlink 

Janus 


Scenario # 



Jlink 

Janus 
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ANOVA for Detection Range Red 


Analysis of Variance Table 
Response: MOE 


Terms added sequentially (first to last) 



Df 

Sum of Sq 

Mean Sq 

F Value 

Pr (F) 

Seen 

2 

19.16607 

9.58304 

588.7363 

0.00000000 

Env 

1 

5.15815 

5.15815 

316.8920 

0.00000000 

Mode 

1 

12.66121 

12.66121 

777.8444 

0.00000000 

Seen:Env 

2 

0.25595 

0.12797 

7.8622 

0.00064909 

Seen:Mode 

2 

3.52387 

1.76194 

108.2451 

0.00000000 

Env:Mode 

1 

0.15869 

0.15869 

9.7494 

0.00230238 

Seen:Env:Mode 

2 

0.08810 

0.04405 

2.7063 

0.07131454 

Residuals 

108 

1.75795 

0.01628 




Interaction plots for scenario:environment, scenariormode, and environment:mode. 
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ANOVA for Kill Range Blue 


Analysis of Variance Table 
Response: MOE 


Terms added sequentially (first to last) 



Df 

Sum of Sq 

Mean Sq 

F Value 

Seen 

2 

14.17509 

7.087543 

111.0971 

Env 

1 

0.43093 

0.430931 

6.7548 

Mode 

1 

2.68719 

2.687186 

42.1216 

Seen;Env 

2 

0.31075 

0.155376 

2.4355 

Seen;Mode 

2 

0.62021 

0.310106 

4.8609 

Env:Mode 

1 

0.02739 

0.027389 

0.4293 

Seen:Env:Mode 

2 

0.18702 

0.093509 

1.4658 

Residuals 

108 

6.88996 

0.063796 



Pr (F) 
0.0000000 
0.0106554 
0.0000000 
0.0923476 
0.0095194 
0.5137113 
0.2354604 


Interaction plots for scenarioienvironment, scenario;mode, and environment:mode. 



Scenario # Scenario # 
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ANOVA for Kill Range Red 


Analysis of Variance Table 
Response: MOE 


Terms added sequentially (first to last) 



Df 

Sum of Sq 

Mean Sq 

F Value 

Seen 

2 

62.94847 

31.47423 

1426.979 

Env 

1 

0.32467 

0.32467 

14.720 

Mode 

1 

1.24480 

1.24480 

56.437 

Seen:Env 

2 

0.33175 

0.16587 

7.520 

Seen:Mode 

2 

0.25323 

0.12661 

5.740 

Env:Mode 

1 

0.30617 

0.30617 

13.881 

Seen:Env:Mode 

2 

0.13178 

0.06589 

2.987 

Residuals 

108 

2.38211 

0.02206 



Pr (F) 

0.00000000 
0.00021033 
0.00000000 
0.00087544 
0.00427329 
0.00031162 
0.05460434 


Interaction plots for scenario:environment, scenarioimode, and environment;mode. 



Scenario # Scenario # 
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ANOVA for Rounds Fired Blue 


Analysis of Variance Table 
Response: MOE 


Terms added sequentially (first to last) 


Df 

Seen 2 
Env 1 
Mode 1 
Seen:Env 2 
Seen:Mode 2 
Env:Mode 1 
Seen:Env:Mode 2 
Residuals 108 


Sum of Sq 
26900478 
1207813 
192240 
1821263 
151142 
110231 
183874 
1740452 


Mean Sq 
13450239 
1207813 
192240 
910631 
75571 
110231 
91937 
16115 


F Value 
834.6258 
74.9482 
11.9290 
56.5073 
4.6894 
6.8402 
5.7049 


Pr (F) 

0.00000000 

0.00000000 

0.00079057 

0.00000000 

0.01114398 

0.01018746 

0.00441275 


Interaction plots for scenariorenvironment, scenario:mode, and environment:mode. 
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ANOVA for Rounds Fired Red 


Analysis of Variance Table 
Response: MOE 


Terms added sequentially (first to last) 



Df 

Sum of Sq 

Mean Sq 

F Value 

Seen 

2 

41699636 

20849818 

943.9545 

Env 

1 

660677 

660677 

29.9115 

Mode 

1 

65 

65 

0.0029 

Seen:Env 

2 

1278612 

639306 

28.9439 

Seen:Mode 

2 

180512 

90256 

4.0863 

Env:Mode 

1 

30401 

30401 

1.3764 

Seen;Env:Mode 

2 

367668 

183834 

8.3229 

Residuals 

108 

2385475 

22088 



Pr (F) 
0.0000000 
0.0000003 
0.9569931 
0.0000000 
0.0194671 
0.2433019 
0.0004348 


Interaction plots for scenarioienvironment, scenario:mode, and environment:mode. 
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fascamSWja faaeam3 swa.ja 1ascam2.hl.ja faacam2.swa.ja (aacam1.hl.ia 


APPENDIX F. NORMAL QUANTILE-QUANTILE PLOTS 


FASCAM.Scen #1,SWA,Janus 


FASCAM.Scen #1,SWA,JUNK 



FASCAM.Scen #1,HL,Janus 


FASCAM.Scen #1,HL,JLINK 



-l.S -1.0 -0.5 0.0 O.S 1.0 1.5 



-1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 


Quanatas at Standard Normal 


Quanblas ol Standard Normal 


FASCAM.Scen #2,SWA,Janus 



-0.5 0.0 O.S 

Quanaiea ol Standard Normal 




FASCAM.Scen #3,SWA,Janus 


FASCAM.Scen #3,SWA,JLINK 
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ch«m3Nj* ch*m3.9waj« chefn2hl.ta cham2 swa )• chaml.hl.(a chami swa.j# 


Chem Kills,Seen #1,SWA,Janus 


Chem Kills,Seen #1,SWA,JLINK 





Chem Kills,Seen #2,HL Janus 



-1.5 -1 0 -0 5 00 OS 10 1.5 

OuanWaa of Standard Normal 


Chem Kilts,Seen #2,HL.JLINK 


(A 



-1.5 -1.0 -0.5 0.0 0.5 1 0 1.5 

Quantlaa of Startdard Normal 



Chem Kills.Seen #3.HL.Janus 


Chem Kills,Seen #3.HL,JLINK 




-1.5 -10 -0.5 0 0 0 5 10 IS 


Quanklaa of Standard Normal 


Otwr>ttlas of Standard Normal 
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l.raHja I•r3.9wai« I•r2hl.^• l•f^.sw*ja l.flMja larlswaia 


LER.Scen #1,SWA,Janus 


LER.Scen #1.SWA,JLINK 



LER.Scen #1 ,HL,Janus LER.Scen #1 .HL.JLINK 



-1 5 -1.0 -05 0.0 0.5 1 0 1.5 -1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 

Quantiles al Standard Normal Ouant'las ol Standard Normal 


LER.Scen #2,SWA,Janus 


LER.Scen #2,SWA.JL1NK 



-1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 



-1.5 -1.0 -0.5 0 0 0.5 1.0 1.5 


Ouantles al Starxlard Normal 


Quantiles of Standard Normal 


LER.Scen #2.HL,Janus LER.Scen #2,HL,JLINK 



-1.5 -1.0 -05 0.0 0.5 1.0 1.5 -1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 

Quantiles ol Standard Normal Quantiles ol Stardard Normal 


LER.Scen #3,SWA,Janus LER.Scen #3,SWA,JL1NK 



Iar3.swa.jl 

0 00 1.00 


-1.5 -1,0 -0.5 0.0 0.5 1.0 1.5 


-1.5 -1.0 -0.5 0.0 05 1.0 1.5 

Quantiles ol Standard Normal 


Ouantles ol Starviard Normal 

LER.Scen #3,HL.Janus 


LER.Scen #3,HL,JLINK 


■=. ®. 

e 

“1 

_ 


-1.5 -1.0 -0.5 0 0 0.5 1.0 1.5 -1.5 -1.0 -0.5 0.0 0.5 1 0 1.5 


Quantiles ol Standard Normal 


Quantiles ol Standard Normal 
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delmgahlbja de«mg3 swa b |e d8tfng2hlb)e ditrrio2 twa.b j» dalmoVhlb.ja dalrngtswabia 


Det Range.Scen #1,SWA,BJue.Janus 


Del Range.Scen #1,SWA.BIue.JLINK 




Det Range.Scen #2,SWA.BIue.Janus 


Det Range.Scen #2.SWA,Blue,JLINK 



•1.5 -1.0 -O.S 00 O.S 1.0 1.S 


-1.5 -1.0 -O.S 0.0 O.S 


1.0 


vs 


Ouanolas d Standard Nomtal 


Ouanblas of Standard Normal 



Det Range.Scen #3,SWA.Blue,Janus 


Det Range.Scen #3,SWA,Blue,JUNK 



•1.5 -1 0 -0 5 0 0 0.5 1.0 1.5 

Quantllas ot Standard Normal 



•1 5 -1.0 -05 00 0.5 1.0 1 5 

Quantiles of Standard Normal 
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detrngS hi r.)a deirngS.swa.f le dalmg2.hl.r.ja datfnga.swa.r.ja datrngl.hl r.|a detrngl.swa f .ja 


Det Range,Seen #1,SWA,Red,Janus 



•0.5 0.0 0.5 

Quantiles of Standard Normal 

Det Range,Scen #1,HL,Red,Janus 



Quantiles of Standard Normal 


Det Range,Scen #2,SWA,Red,Janus 



-1.0 -0.5 0.0 0.5 1.0 

Quantiles of Standard Normal 


t r 


Det Range,Seen #1,SWA,Red.JLINK 



-1.0 -0.5 0.0 0.5 1.0 

Quantiles of Standard Normal 

Det Range,Seen #1,HL,Red,JLINK 



-1.0 -0.5 0.0 0.5 1.0 

Quantiles of Standard Normal 

Det Range.Seen #2,SWA,Red,JLINK 



-0.5 0.0 0.5 

Quantiles of Standard Normal 


Det Range.Seen #2,HL,Red,Janus 


Det Range.Seen #2.HL.Red.JLINK 



— ® J 

s «» 

£ ^ 

o 

^ CM 


^-1.0 -0 5 0.0 0.5. 1.0 1.5 

Quantiles of Standard Normal 

Det Range.Seen #3.SWA.Red.Janus 

oi 

•1.5 -1 0 -0.5 0.0 0 5 1.0 1.5 

Quantiles of Standard Normal 

Det Range.Seen #3,SWA,Red,JLINK 


s. 

v> ^ 

ri 


_ 

detrn 

1.98 



Quantiles of Standard Normal 


Det Range.Seen #3,HL,Red,Janus 



-1.0 -0.5 0.0 0.5 

Quantiles of Stendard Normal 


-1.0 -0 5 0.0 0.5 1.0 

Quantiles of Standard Normal 

Det Range.Seen #3,HL.Red.JLINK 



-1.5 -1.0 -0.5 0.0 0.5 

Quantiles of Standard Normal 
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killrngS hi b )a ktlIrngS swa b.)a killrng2 hi b |a killmg2 awab ja kjllmg1.hl.b.|a killmgl 3Mra.b.ja 


Kill Range.Scen #1,SWA,Blue,Janus 


Kill Range,Scen #1,SWA.BIue,JLINK 



^ o 

i ^ 

6 o 


-IS -1.0 -05 00 0.5 1.0 1 5 


-1 5 1 0 -0 5 0 0 0 5 1.0 1 5 

Ouanblas ol Standard Normal 


Ouantlas ol Standard Normal 

Kill Range.Scen #1,HL,Blue,Janus 


Kill Range.Scen #1,HL.Blue.JLINK 

^^ ^ 

a> 


^— 

I n. 


^ 

1 . 

^- 


S ci 


1.5 -1 0 -05 0.0 05 1.0 1.5 


-1.5 1.0 -05 0.0 05 1.0 1.5 

Ouanblas ct Starxlard Normal 


Quanblas ol Standard Normal 

Kill Range.Scen #2,SWA,Blue,Janus 


Kill Range.Scen #2,SWA.BIue.JLINK 


1 



5 

E 



1.5 -1.0 -O.S 0.0 0.5 10 1.5 

Ouanblas o( Standard Normal 


•1 5 .1.0 -0.5 0.0 0 5 1 0 1 5 

Quanbtas o( Standard Normal 



Kill Range.Scen #3.SWA,Blue,Janus 



1.5 -1 0 -0.5 0 0 0.5 1.0 1 5 

Ouantilaa o( Standard Normal 


Kill Range.Scen #3,HL,Blue,Janus 


1.5 1.0 -0.5 0.0 0.5 1.0 1.5 

Ouanliloa o1 Startdard Normal 




•IS -1.0 -0.5 0.0 0.5 1.0 1.5 

Quantilas ol Standard Normal 


Kill Range.Scen #3.HL,Blue,JLINK 



-1 5 -1.0 -05 0.0 05 1.0 1.5 

Qtianttlas ol Standard Normal 
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killmo 3 .h(.r.i« killmg3.svMi.r.)a killmg2.hl.f.j. killmg^ swa.r.)a Wlm 3 l.Wf.ja killmgl swa f.(a 


Kill Range.Scen #1,SWA,Red,Janus 


Kill Range.Scen #1 .SWA.Red.JLINK 



Ouanb'las d Standard Nomial 


Kill Range.Scen #1,HL,Red,Janus 



•VO -O.S 0.0 0.S 

Ouanblas d Standard Normal 

Kill Range.Scen #2,SWA,Red.Janus 



- 1.0 -O.S 0.0 O.S 1.0 
Ouanblas d Standard Normal 

Kill Range.Scen #1,HL.Red,JLINK 



•O.S 0.0 0.5 

Ouanblas d Standard Normal 

Kill Range.Scen #2,SWA.Red,JLINK 






i . 

'S. CM 



-1.5 -1.0 -0.5 0.0 0.5 

Ouanblas d Standard Normal 


- 1.0 -0.5 0.0 0.5 

Ouanblas d Starxiard Normal 


- 1.0 -0.5 0.0 0.5 1.0 

Ouanblas d Standard Normal 

Kill Range.Scen #3,HL,Red,Janus 


- 1.0 -0.5 0.0 0.5 1.0 

Ouanblas d Standard Normal 

Kill Range.Scen #3,HL.Red.JLINK 



•1.0 -0.5 0.0 0.5 

Ouanblas d Standard Normal 


Kill Range.Scen #2,HL,Red,Janus 


Kill Range.Scen #2,HL,Red,JLINK 


o 

■ 1 

1 



" s 


-1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 

Ouanblas d Starxiard Normal 

Kill Range.Scen #3, SWA, Red .Janus 


“’’trS -1.0 -0.5 0.0 0.5 1.0 1.5 

Ouanblas d Standard Normal 

Kill Range.Scen #3,SWA,Red,JLINK 


I - 

^_ 


kilim 

1.30 




Ouanblas d Slarxlard Normal 
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rnd$3hlb|» rnd*3 swa b )• rnds2.hlb|a rnds2.5wabja rndsl .ht.b.ja rndst.swab ja 


Rnds Fired,Seen #1 .SWA.BIue,Janus 


Rnds Fired,Seen #1,SWA,Blue,JLINK 




-1.5 -10 -0.5 0.0 OS 1.0 1.S 


Ouamilss of Standard Normal 


QuanUlas of Standard Normal 


Rnds Fired,Seen #1,HL,Blue,Janus 



OuantHas of Starvlard Normal 


Rnds Fired,Seen #1 ,HL,Blue,JLINK 



-1 5 -1.0 -0.5 0.0 0.5 1.0 1.5 

Ouantllas of Standard Normal 


Rnds Fired,Seen #2,SWA,Blue,Janus 



-1.5 -1 0 -0.5 0.0 0.5 1.0 1 5 


Rnds Fired,Seen #2,SWA,Blue,JLINK 



-1 5 -1.0 -05 00 0.5 1.0 1.5 


OuantHas of Standard Normal 


QuantUas of Standard Normal 


Rnds Fired.Seen #2,HL.Blue,Janus 


Rnds Fired.Seen #2.HL,Blue,JLINK 



OuantHas of Standard Normal 


Ouanlilas of Standard Normal 


Rnds Fired.Seen #3.SWA,Blue,Janus 



-1 5 -1.0 -0,5 0.0 0.5 1.0 1 5 


Rnds Fired.Seen #3,SWA,Blue,JLINK 



-1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 


Ouanittas of Standard Nonna! 


Ouanlilas of Standard Normal 
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APPENDIX G. KOLMOGOROV-SMIRNOV GOODNESS OF FIT RESULTS 


Highlighted portions are those MOPs which produce p-values<0.05 ((X=0.05), and 
thus reject the null hypothesis that the sample comes from a Normal distribution. 


Scenario#!, SWA 


FASCAM 

Chem Kills 

LER 

Mode 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

P-Value 

.60 

.75 

,04 

.03 

.44 

.73 

Scenario #1,HL 


FASCAM 

Chem Kills 

LER 

Mode 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

P-Value 

.92 

.63 

.47 

.42 

.43 

.84 

Scenario^ SWA 


FASCAM 

Chem Kills 

LER 

Mode 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

P-Value 

.48 

.84 

.20 

.004 

.79 

.89 

Scenario #2, HL 


FASCAM 

Chem Kills 

LER 

Mode 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

P-Value 

.14 

.08 

.91 

.44 

.74 

.99 

Scenario #3, SWA 


FASCAM 

Chem Kills 

LER 

Mode 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

P-Value 

.46 

D04 

.51 

,01 

.59 

.95 

Scenario #3, HL 


FASCAM 

Chem Kills 

LER 

Mode 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

P-Value 

.60 

.08 

,001 

.93 

.38 

.93 


Ill 































































Scenario#!, SWA 


Det Rng Blue 

Det Rng Red 

Kill Rng Blue 

KiU Rng Red 

RndsRiedBlue 

Rnds Fired Red 

Mode 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

P-Value 

.98 

.93 

51 

.25 

.10 

.84 

.94 

.44 

•63 

.24 

.93 

.89 

Scaiario#l,HL 


Det Rng Blue 

Det Rng Red 

KiU Rng Blue 

KiU Rng Red 

RndsRiedBlue 

Rnds Fired Red 

Mode 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

P-Value 

.97 

.65 

.56 

.92 

.99 

.95 

.99 

.75 

.19 

.45 

.94 

.98 

Scenario #2, SWA 


Det Rng Blue 

Det Rng Red 

KiM Rng Blue 

KiU Rng Red 

RndsRiedBlue 

Rnds Fired Red 

Mode 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

P-Value 

.41 

.21 

.78 

.95 

.08 

.10 

.92 

.91 

.97 

.19 

.40 

.85 

Scenario #2, HL 


Det Rng Blue 

Det Rng Red 

Kill Rng Blue 

KiU Rng Red 

RndsRiedBlue 

Rnds Fired Red 

Mode 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

P-Value 

.78 

.58 

.96 

.96 

.46 

.85 

.82 

.43 

.80 

.58 

.97 

.81 

Scenario #3, SWA 


Det Rng Blue 

Det Rng Red 

KiU Rng Blue 

KiU Rng Red 

RndsRiedBlue 

Rnds Fired Red 

Mode 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

P-Value 

.68 

.52 

.92 

.64 

.98 

26 

.98 

.76 

.50 

.92 

.82 

.90 

Scenario #3, HL 


Det Rng Blue 

Det Rng Red 

KiU Rng Blue 

KiU Rng Red 

RndsRiedBlue 

Rnds Fired Red 

Mode 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

P-Value 

.47 

-97 

.33 

.87 

.70 

.44 

.97 

.98 

29 

.94 

.97 

.91 
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APPENDIX H. F-TEST FOR COMMON VARIANCE RESULTS 


Highlighted portions are those MOPs which produce p-values<0.05 ((x=0.05), and 
thus reject the null hypothesis that the two samples share a common variance. 


Scenario#!, SWA 


FASCAM 

Chem Kills 

LER 

Mode 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

P-Value 

.31 

.29 

.97 

Scenario #1,HL 


FASCAM 

Chem Kills 

LER 

Mode 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

P-Value 

.045 

.24 

.10 

Scenario #2, SWA 


FASCAM 

Chem Kills 

LER 

Mode 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

P-Value 

.46 

.051 

.14 

Scenario #2, HL 


FASCAM 

Chem Kills 

LER 

Mode 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

P-Value 

.003 

.15 

.11 

Scenario #3, SWA 


FASCAM 

Chem Kills 

LER 

Mode 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

P-Value 

.01 

.01 

.00 

Scenario #3, HL 


FASCAM 

Chem Kills 

LER 

Mode 

Janus 

JUnk 

Janus 

JLink 

Janus 

JLink 

P-Value 

.045 

.00 

.23 
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Scenario#!, SWA 


Del Rng Blue 

Det Rng Red 

KiU Rng Blue 

KUl Rng Red 

Rnds Bred Blue 

Rnds Rred Red 

Mode 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

Janus 

Jlink 

P-Value 

.39 

.73 

,00 

37 

.00 

.76 

Scenario #1^HL 


Det Rng Blue 

Det Rng Red 

KUl Rng Blue 

KUl Rng Red 

Rnds Hied Blue 

Rnds Rred Red 

Mode 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

Janus 

Jlink 

P-Value 

.24 

.13 

.66 

.23 

.00 

.00 

Scenario #2, SWA 


Det Rng Blue 

Det Rng Red 

KiU Rng Blue 

KUl Rng Red 

Rnds Rred Blue 

Rnds Rred Red 

Mode 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

Janus 

Jlink 

Janus 

Jlink 

P-Value 

.12 

.19 

.50 

.17 

.82 

.01 

Scaiario#2,HL 


Rng Blue 

Det Rng Red 

KUl Rng Blue 

KUl Rng Red 

Rnds Rred Blue 

Rnds Rred Red 

Mode 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

Janus 

Jlink 

Janus 

Jlink 

P-Value 

.45 

.00 

.21 

31 

.94 

.99 

Scenario #3, SWA 


Det Rng Blue 

Det Rng Red 

KUl Rng Blue 

KiURngRed 

Rnds Rred Blue 

Rnds Rred Red 

Mode 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

Janus 

Jlink 

Janus 

Jlink 

P-Value 

.25 

.98 

,01 

.98 

.00 

.17 

Scenario #5, HL 


Det Rng Blue 

Det Rng Red 

KUl Rng Blue 

KUl Rng Red 

Rnds Rred Blue 

Rnds Rred Red 

Mode 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

Janus 

Jlink 

Janus 

JUnk 

P-Value 

.84 

.22 

,02 

.05 

.12 

.78 
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APPENDIX 1. /-TEST RESULTS 


Highlighted portions are those MOPs which produce p-values<0.0083, 
i^new ~ ^oid / 6 = 0.0083) and thus reject the null hypothesis that the two sample means are 
statistically similar. Also, the = 99.17 percent confidence interval (Cl) is provided to 

show the extent to which the sample means are similar (dissimilar). All MOPs failing to 
reject the null hypothesis (p-value>0.0083) produce CIs which include zero. 
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Scenario #1,SVVA 


FASCAM 

Chem Kills 

LER 

Mode 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

Cl 

(-1.18,2^8) 


(.66,1.38) 

p-value 


na 

.00 

Scenario #1, HL 


FASCAM 

Chem Kills 

im 

Mode 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

a 


(7.81,11.79) 

(-.27,58) 

p-value 

na 

.00 

30 

Scenario #2, SWA 


FASCAM 

Chem Kills 

UER 

Mode 

Janus 

JLink 

Janus 

JLink 

Janus 

JUnk 

Cl 

(-1.14,2.74) 


(-.09,.49) 

p-value 

.24 

na 

.05 

Scenario #2, HL 


FASCAM 

Chem Kills 

UER 

Mode 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

Cl 



(-.72,.03) 

p-value 

na 

na 

.014 

Scenario #3>, SWA 


FASCAM 

Chem Kills 

UER 

Mode 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

p-value 

na 

na 

na 

Scenario #3, HL 


FASCAM 

Chem Kills 

LER 

Mode 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

Cl 



(-.37,1.88) 

p-value 

na 

na 

.06 
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Scenario#!, SWA 


Det Rng Blue 

Det Rng Red 

KiU Rng Blue 

Kill Rng Red 

Rnds Bred Blue 

Rnds Bred Red 

Mode 

Janus 

JUnk 

Janus 

JUnk 

Janas 

JUnk 

Janus 

JUnk 

Janas 

JUnk 

Janus 

JUnk 

Cl 

(-.60,-.37) 

(-1.13,-.71) 

(-.66,.02) 

(-.54,.03) 

(17.52,87.48) 

(-129.14,41.26) 

P-ValiK 

.00 

,00 

.02 

.02 

,00 

.00 

Scenario#!, HL 


Det Rng Blue 

Det Rng Red 

Kill Rng Blue 

Kill Rng Red 

Rnds Bred Blue 

Rnds Bred Red 

Mode 

Janus 

JUnk 

Janus 

JUnk 

Janus 

JUnk 

Janus 

JUnk 

Janus 

JUnk 

Janus 

JUnk 

Cl 

(-.74.-.50) 

(-1.08,-.68) 

(-.91,-.30) 

(-.53,-.25) 

(19.14,89.26) 

(-60.62,476.82) 

P-Value 

,00 

,00 

,00 

.00 

,00 

.03 

Scenario #2, SWA 


Det Rng Blue 

Det Rng Red 

Kill Rng Blue 

Kill Rng Red 

Rnds Bred Blue 

Rnds Pared Red 

Mode 

Janus 

JUnk 

Janus 

JUnk 

Janus 

JUnk 

Janus 

JUnk 

Janus 

JUnk 

Janus 

JUnk 

Cl 

(-.65,-.26) 

(-1.21,-.85) 

(-.90,.22) 

(-.28,.28) 

(2.88,39.52) 

(-104.78,61.38) 

P-ValiK 

,00 

,00 

.09 

.99 

.003 

.45 

Scenario #2, HL 


Det Rng Blue 

Det Rng Red 

Kill Rng Blue 

Kill Rng Red 

Rnds Bred Blue 

Rnds Pared Red 

Mode 

Janus 

JUnk 

Janus 

JUnk 

Janus 

JUnk 

Janus 

JUnk 

Janus 

JUnk 

Janus 

JUnk 

Cl 

(-.70,-.47) 

(-.97,-.51) 

(-.70,.07) 

(-34,-.23) 

(-28.42,24.42) 

(-71.46,294.46) 

P-Value 

,00 

.00 

.03 

.00 

.83 

.09 

Scenario #3, SWA 


Det Rng Blue 

Det Rng Red 

Kill Rng Blue 

Kill Rng Red 

Rnds Bred Blue 

Rnds Pared Red 

Mode 

Janus 

JUnk 

Janus 

JUnk 

Janus 

JUnk 

Janus 

JUnk 

Janus 

JUnk 

Janus 

JUnk 

Cl 

(-.05,.04) 

(-.26,-.18) 

(-.22,-.08) 

(-.14,.03) 

(109.48,587.12) 

(-111.99,125.99) 

P-Valuc 

.63 

,00 

,00 

.07 

,00 

.86 

Scenario #3, HL 


Det Rng Blue 

Det Rng Red 

Kill Rng Blue 

KiU Rng Red 

Rnds Bred Blue 

Rnds Pared Red 

Mode 

Janus 

JUnk 

Janus 

JUnk 

Janus 

JUnk 

Janus 

JUnk 

Janus 

JUnk 

Janus 

JUnk 

Cl 

(-.12,-.03) 

(-.16,-.07) 

(-.13,-.02) 

(-.28,.001) 

(-324.35,336.55) 

(-550.96,93.96) 

P-Valu( 

.00 

.00 

.001 

.009 

.96 

.05 
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APPENDIX J. WILCOXEN TEST RESULTS 


Highlighted portions those MOPs which produce p-values<0.0083, (/ 6 = 0.0083) 
and thus reject the null hypothesis that the two sample means are statistically similar. 


Scenario#!, SWA 


FASCAM 

Chem Kills 

LER 

Mode 

Janus 

JUnk 

Janus 

JLink 

Janus 

JLink 

p-value 

.29 

.000 

.000 

Scenario #1,HL 


FASCAM 

Chem Kills 

LER 

Mode 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

p-value 

m 

.000 

.47 

Scenario #2, SWA 


FASCAM 

Chem Kills 

LER 

Mode 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

p-value 

.26 

.000 

.05 

Scenario #2, HL 


FASCAM 

Chem Kills 

LER 

Mode 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

p-value 

.001 

.000 

.02 

Scenario #3, SWA 


FASCAM 

Chem Kills 

LER 

Mode 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

p-value 

.08 

.000 

.001 

Scenario #3, HL 


FASCAM 

Chem Kills 

LER 

Mode 

Janus 

JLink 

Janus 

JLink 

Janus 

JLink 

p-value 

.49 

.000 

m 
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Scenario#!, SWA 


Det Rng Blue 

Det Rng Red 

Kill Rng Blue 

Kill Rng Red 

RndsRredBlue 

RndsRredRed 

Mode 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

p-value 

,000 

.000 

.0015 

.02 

.000 

.000 

Scenario#!, HL 


Det Rng Blue 

Det Rng Red 

Kill Rng Blue 

Kill Rng Red 

RndsRredBlue 

RndsRredRed 

Mode 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

p-value 

.000 

.000 

.000 

.000 

.000 

.08 

Scenario #2, SWA 


Det Rng Blue 

Det Rng Red 

Kill Rng Blue 

Kill Rng Red 

RndsRredBlue 

RndsRred Red 

Mode 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

p-value 

.000 

.000 

.01 

.74 

.005 

.12 

Scenario#2,HL 


Det Rng Blue 

Det Rng Red 

Kill Rng Blue 

Kill Rng Red 

RndsRredBlue 

RndsRredRed 

Mode 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

p-value 

.000 

.000 

.04 

.000 

.97 

.08 

Scenario #3, SWA 


Det Rng Blue 

Det Rng Red 

Kill Rng Blue 

Kill Rng Red 

RndsRredBlue 

RndsRredRed 

Mode 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

p-value 

.48 

.000 

.000 

.11 

.000 

.94 

Scenario #3, HL 


Det Rng Blue 

Det Rng Red 

Kill Rng Blue 

Kill Rng Red 

RndsRredBlue 

RndsRredRed 

Mode 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

Janus 

JUNK 

p-value 

.000 

.000 

.0011 

.009 

.48 

.06 
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